/LLM-security-study

An Investigation into Misuse of Java Security APIs by Large Language Models

Primary LanguageJava

An Investigation into Misuse of Java Security APIs by Large Language Models

This repository contains materials related to the study: "An Investigation into Misuse of Java Security APIs by Large Language Models" accepted by the 19th ACM ASIA Conference on Computer and Communications Security (ASIACCS 2024).

Pre-print of the Paper

Pre-print available on arXiv

Citation

To cite this work, please use the following format:

@inproceedings{mousavi2024investigation,
  title={An Investigation into Misuse of Java Security APIs by Large Language Models},
  author={Mousavi, Zahra and Islam, Chadni and Moore, Kristen and Abuadbba, Alsharif and Babar, Muhammad Ali},
  booktitle={19th ACM ASIA Conference on Computer and Communications Security (ASIACCS 2024)},
  year={2024}
}

Contents

  • raw_responses directory: Contains responses obtained from ChatGPT using security programming tasks as prompts. This directory is organized with separate subdirectories for each security API, each containing 30 responses received for each task of the respective API.
  • src directory: Includes extracted code snippets from the responses obtained. This directory is organized with separate subdirectories for each security API, each containing programs obtained for each task of the respective API.
  • Analysis_results.xlsx: Presents the results derived from the analysis of each individual response obtained during the study.

For further details, please refer to the paper.