Explicit Syntactic Guidance for Neural Text Generation

Yafu Li^†^‡, Leyang Cui^¶, Jianhao Yan^†^‡, Yongjing Yin^†^‡,

Wei Bi^¶, Shuming Shi^¶, Yue Zhang^‡

^† Zhejiang University, ^‡ Westlake University, ^¶ Tencent AI Lab

A syntax-guided generation schema that generates the sequence guided by a constituency parse tree in a top-down direction (See paper for details).

Environment Setup

To set up the environment, clone the project and run the following script:

bash setup_env.sh # run under SyntacticGen directory

Toy Example

To demonstrate the process, we have prepared a toy example that includes data preparation, model training, and model inference. Please ensure to specify the "PROJECT_PATH" in each script according to your actual project path.

# Set the PROJECT_PATH variable to your directory

PROJECT_PATH=$YOUR_DIRECTORY

# Data Preparation: This script builds training triplets using source text data, target text data, and parsing results.

bash $PROJECT_PATH/shell/prepare_data.sh

# Model Training: Train the neural decoder on the training triplets.

bash $PROJECT_PATH/shell/train.sh

# Model Inference: Use the trained model for inference. Structural beam search is enabled if beam size is larger than 1.

bash $PROJECT_PATH/shell/infer.sh

We provide the model for paraphrase generation in our paper at Google Drive.

Arguments

Training

Argument	Description
--arch	model architecture, use ''at_tree_attn_nonshare'' by default
--task	training procedure is simply a seq2seq task, i.e., ''translation''
--encoder-layers	Number of layers for the source text encoder
--decoder-layers	Number of layers for the target text decoder
--ctx-encoder-layers	Number of layers for the syntax context encoder

Inference

Argument	Description
--beam	Beam size for structural beam search
--prev-score-ratio	Weight on previous decisions when re-ranking newly updated beam candidates (α)
--cur-score-ratio	Weight on current-level decisions (usually 1-α)

Citation

@misc{li2023explicit,
      title={Explicit Syntactic Guidance for Neural Text Generation}, 
      author={Yafu Li and Leyang Cui and Jianhao Yan and Yongjing Yin and Wei Bi and Shuming Shi and Yue Zhang},
      year={2023},
      eprint={2306.11485},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}