/parser

Primary LanguagePython

parser

Codes for dependency parsing, including following models:

  1. Deep Biaffine Attention for Neural Dependency Parsing (Dozat and Manning, 2016) .

  2. (developing) Token-to-Token Mrc Dependency parsing

    • fix child score bug
    • add biaffine struct/multi-layer
    • change bracket special token from [SEP] to other [unused]
    • use mst, fix multi-gpu mst
    • change type-id of [SEP]

  3. (developing) Span-to-Span MRC Dependency parsing

    • joint training stage1/stage2
    • 后面多加1-n层transformer
    • 加label smoothing
    • 做实验，在dp decode时候把gt推进去是否会变好

Requirements

  • python>=3.6
  • pip install -r requirements.txt

Dataset

We followed this repo for PTB/CTB data preprocessing.

Reproduction

1. Deep Biaffine Attention

Train

See scripts/biaf/biaf_ptb.sh Note that you should change MODEL_DIR and BERT_DIR to your own path.

Evaluate

See biaf_evaluate.py Note that you should change HPARAMS and CHECKPOINT to your own path.

2. Token-to-Token MRC (Developing)

Train

See scripts/t2t/train.sh Note that you should change MODEL_DIR and BERT_DIR to your own path.

Evaluate

See parser/t2t_evaluate.py

3. Span-to-Token MRC (Developing)

Train

  • proposal model: scripts/s2t/pengcheng_ptb_proposal.sh
  • query model: scripts/s2t/pengcheng_ptb_query.sh

Evaluate

See parser/s2t_evaluate_dp.py

4. Span-to-Span MRC (Developing)

Train

  • proposal model: scripts/s2s/*/proposal.sh
  • s2s model: scripts/s2t/*/s2s.sh

Evaluate

Choose the best proposal model and s2s model independently, and run

parser/s2s_evaluate_dp.py \
--proposal_hparams <your best proposal model hparams file> \
--proposal_ckpt <your best proposal model ckpt> \
--s2s_ckpt <your best s2s query model hparams file> \
--s2s_hparams <your best s2s query model ckpt> \
--topk <use topk spans for evaluating>

TODO

  • refactor config/argparser hyper-parameters
  • refactor functions that use roberta
  • refactor usage of from_pretrained of pretrained bert/roberta, move it into model
  • ddp sampler may cause training data in same order between different epoch