mahimanzum/FixEval
We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of execution based evaluation compared to suboptimal match based evaluation metrics like BLEU, CodeBLEU, Syntax Match, Exact Match etc.
PythonMIT
Stargazers
- bqcuongHamburg, Germany
- DehengYang
- fly51flyPRIS
- gmftbyGMFTBYBeijing Institute of Technology
- GuodongFan
- haiasd
- HFConsultant
- HickeyHsu
- Horizon-MBeijing, China
- L1aoXingyuBeijing, China
- nashid@UBC
- NTDXYGNanjing University of Aeronautics and Astronautics
- roy029こたつ
- SandalotsVolcanak
- scrisBUPT, Beijing
- SymbolkBeijing
- ZanejinsPeking University
- zfj1998City University of Hong Kong
- zscBeijing