Reflexion Paper might not work
Closed this issue · 1 comments
isamu-isozaki commented
Hi! Thanks for the repo. There was a paper from Deepmind "Large Language Models Cannot Self-Correct Reasoning Yet" that seemed to say that Reflexion(which is listed in this repo) or any self-correction methods up to this point don't seem to work. The criticism for Reflexion in particular was that the authors used oracle labels(ground truth) for evaluation and I think they claimed removing that removed the performance gains. I think technically the part the paper agreed on is that external feedback can be useful.