Several minor issues when running tests on ClassEval.
ztwater opened this issue · 3 comments
Thanks for your contribution with a class-level context-dependent method generation benchmark. Here are some minor problems I found when I inspected the test results. I downloaded this benchmark in August and maybe some of them have been fixed up to now.
-
测试方法中所依赖的python包需要提前安装,建议可以给一个list,否则会导致运行测试时大量的NameError。
-
ClassEval_5_display
返回值应该是string而非None -
ClassEval_44_format_line_html_text
测试代码中多了from benchmark_code.HtmlUtil import HtmlUtil -
ClassEval_49_search_jobs
:param criteria: The requirements of the position,list.
实际上输入是字符串,e.g., 'skill1' -
ClassEval_50
类名是JSONProcessor,实际文件命名为JSONHandler -
ClassEval_52_lemmatize_sentence
"solution_code"删去nltk download语句,下载nltk对应包,放到环境中,防止测试超时的问题 -
ClassEval_60_create_table
方法描述中的字段名写错,'author name'应该是'theater_name' -
ClassEval_69_extract_text_from_pdfs
方法的测试代码少了一个测试类 -
ClassEval_94_display_items
:return: If the vending machine is empty, returns False, otherwise, returns a list of the products in the vending machine, list.
实际测试用例写的是返回 string
非常感谢您的反馈!我们会仔细检查这些问题并且重新发布升级后的benchmark以及所需的list
我们已经修复您提到的问题,重新发布了1.1.0版本的classeval数据,并且更新了requirements
感谢你们的迅速回复和处理!ClassEval是一个对领域很有价值的benchmark,期待它的影响力越来越大。