what is the "lift_ratio" in the file config?
Closed this issue · 4 comments
as title.
In the peft_wrapper.py, some code of the function "prepare_inputs_for_generation_peft" confuses me. Can you please tell me what does these lines of code want to do?
Especially codes in the else block. I think it will not be conducted in any case. Am I right?
I have a question about the aggregation process of MAC. During inference, MAC aggregates document information into the compressed question prompts, and I wonder, have you done any experiments that only use the compressed question prompts to form the extra p-tuningv2 model parameters without any documents? I would like to know your experiment results in this setting because I tried to do so and it seems that the documents provide limited information to the compressed question prompts.
Hi, thank you for your interest in our paper,
I did not see the issue. lift_ratio is the backpropagation dropout ratio that is used in the paper.
For your question, during development, we tested on distilgpt2, and it worked well with the documents. (by only using the question it show some degradation, but not that much as the overall score is low for this model size).
I am on a holiday for quite a while, and happy to help any issue afterwards! I usually don't see the github so feel free to reach out using my email :) jihoontack@gmail.com
Best,
Jihoon
In the peft_wrapper.py, some code of the function "prepare_inputs_for_generation_peft" confuses me. Can you please tell me what does these lines of code want to do?
Especially codes in the else block. I think it will not be conducted in any case. Am I right?
Still confused now.