techandy42/bug_in_the_code_stack_v2

Can LLMs find bugs that compilers can't?: A benchmark for measuring LLMs' capabilities in debugging large source code.

BICS v2

Environment Set-up

• Create a new Conda environment and install packages with the following commands:

conda create --name bics_v2 python=3.9
conda activate bics_v2
conda install python-dotenv pandas
pip install litellm

Datasets

• techandy42/multi_lang_leetcode

Google Drive

• All Colab notebooks can be found at: https://drive.google.com/drive/folders/1QtuJ3lAICllWqywlmbNOQ7sPgcoareUH?usp=sharing