Problem with generating longer summaries
v-zmiycharov opened this issue · 3 comments
Hi there! Great job on this fantastic implementation!
I am trying to use PreSumm algorithm and fine tune it on dataset which has average full text length of 10000 tokens and summary of 800 tokens.
I have 1 question:
- Do I need to set max_pos parameter to >10000 in order to read the entire content of full documents
And 1 issue: - Even though I set max_length parameter to 1200 the algorithm always generates summaries with about 120 tokens which reduces recall. How do I change this?
I have the same Questions.
@nlpyang Can you kindly help us in this regard? If we want to summarize a document with 3000 token, we just need to set max-pos = 3000 in order to consider the whole document?
I have the same Questions.
@nlpyang Can you kindly help us in this regard? If we want to summarize a document with 3000 token, we just need to set max-pos = 3000 in order to consider the whole document?
Since the max_pos of BERT-base is 512, so if you have longer input, I think you have to modify the model yourself, like using a sliding window.
No, but the documentation says that if you have a longer input such as 800 so we can use max-pos = 800.
And thus my Q is if I pass 4000 then it will consider first 4000 words?