CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. Defect detection notebook tries to achieve the results showcased in the original paper of CodeXGLUE for detecting any vulnerable C code using CodeBERT model with RoBERTa baseline.
CodeXGLUE : https://arxiv.org/abs/2102.04664
Defect Detection
| Test Accuracy | RoBERTa | CodeBERT |
|---|---|---|
| CodeXGLUE | 61.05 | 62.08 |
| Notebook | 53.3 | 62.40 |