generate_texts on wbmodel ignores generation parameters and stopping critera.
Opened this issue · 0 comments
rvashurin commented
What title says.
This causes foundation models to generate lots of unnecessary text, introduces potential discrepancy between sampling and greedy generation with generate, and possible other less obvious problems. Problematic behavior can be reproduced by only having blackbox_sample_texts in required stats with no sample_texts on any foundation model with few-shot continuation prompt.
This calls for some streamlining of generation when using white box. Do we really need a separate generation method to pretend we are black-box when calculating things like semantic matrix on WB model? Can we call self.generate instead of self.model.generate in generate texts?
@ArtemVazh @cant-access-rediska0123 your thoughts?