Below is an annotated map of the directory structure of this repository.
.
├── scripts................................ Scripts to exactly reproduce all experiments presented in our paper.
│ └── <dataset>.......................... One directory for each dataset (CodeXGLUE, VulDeePecker).
│ └── <technique>.................... One directory for each ML technique (VulBERTa, CoTexT, PLBart)
│ ├── run.py..................... Compute experimental results for selected ML technique and dataset.
│ └── run_at.py.................. Compute adversarial training results for selected ML technique and dataset. Only for CodeXGLUE.
│
├── datasets............................... All datasets that we use in the experiments for the paper + our own created dataset VulnPatchPairs.
│ └── README.md.......................... Instructions for downloading all datasets used in this repository.
│
├── models................................. All pretrained models that are not downloaded in the training scripts.
│ └── README.md.......................... Instructions for downloading all models used in this repository.
│
├── plots.................................. Generates all plots shown in the paper.
│ ├── generate_plots.py.................. Script that generates all plots from the experimental results.
│ └── plots.............................. The produced plots and tables that can be found in the paper.
│
├── additional_experiments................. Additional experiments presented in the paper.
│ └── naturalness........................ Additional experiments on naturalness of transformations.
│ └── run.py......................... Run additional experiment on naturalness.
│
├── install_requirements.sh................ Script to install Python environment and required packages.
├── requirements.txt....................... All Python packages that you need to run the experiments.
│
├── run_experiments.sh..................... Script to reproduce all experiments presented in our paper.
│
└── README.md
Anaconda is an open-source package and environment management tool for Python. Instructions for Installation can be found here.
We assume that you have Anaconda installed.
Running the following script from the root directory of this repository creates a virtual environment in Anaconda, and installs the required Python packages.
bash install_requirements.sh
Activate the environment with the following command.
conda activate LimitsOfMl4Vuln
Go to datasets/README.md and follow the instructions to download all datasets needed to run our experiments.
Go to models/README.md and follow the instructions to download all models needed to run our experiments.
Run
bash run_experiments.sh
to reproduce all experimental results presented in our paper. The script also serves as an entry point into the scripts for the different experiments.