斯坦福小镇

[2642543078]

文中的感知和认知 与现在开源的项目斯坦福小镇有何区别

[chenllliang]

Hello,
I'd like to point out a key distinction between the PCA-EVAL's Perception and Cognition and the agents discussed in the Stanford paper. In PCA-EVAL, the Perception and Cognition are inherently multimodal, accommodating multiple forms of input like visual and textual. In contrast, the agents in the Stanford paper are exclusively designed to process textual input. This distinction is crucial when comparing their functionalities and capabilities.

[2642543078]

你的意思是他们的一个可以容纳多种形式 而另外一个只可以容纳一种处理的文本 我问下感知和认知反思都是来表示什么 都是干嘛的

[chenllliang]

In our paper,

• Perception-Score: Evaluate whether the agent captures the key concepts related to the question in the observation.
• Cognition-Score: Evaluate whether the agent makes correct deduction with perception and world knowledge to the final action.
• Action-Score: Evaluate whether the agent chooses the correct action.

You can refer the section 3.1 in our paper for more details.

In my opinion, Reflection(not covered in our paper) in LLM is about learning from judging the past experience or examples.