This project "An Automated Evaluation Framework for Graph Database Query Generation Leveraging Large Language Models," is designed to streamline the process of evaluating graph database query generation using large language models. By automating various aspects of query generation and evaluation, this framework aims to enhance the efficiency and accuracy of graph database query development.
This file provides the necessary login information for accessing the ChatGPT API. ChatGPT is utilized within the framework to generate natural language queries based on given parameters and templates.
neo4j_client.py contains the login information required to connect to a Neo4j graph database. Neo4j is a popular choice for graph databases due to its powerful querying capabilities and scalability.
design_template.json serves as the raw data template for query generation. This file contains the structure and specifications required for generating queries. Users can customize this template according to their specific graph database schema and requirements.
The data_preparation.py script is responsible for preparing the data required for query generation. It processes the raw data template (design_template.json) and transforms it into a format suitable for input into the query generation process.
query_generation.py is the main script of the framework. It utilizes the processed data from data_preparation.py and leverages the ChatGPT model to generate natural language queries. These queries are then executed against the Neo4j database, and the results are evaluated based on predefined metrics.
To utilize this framework, follow these steps:
- Ensure you have conda installed on your system.
- Create a new Python 3.11 environment using conda by running the following command in your terminal:
conda create -n graph_query_env python=3.11- Activate the newly created environment:
conda activate graph_query_env- Install the required dependencies listed in requirements.txt by running:
pip install -r requirements.txt- Provide the necessary login credentials in chatgpt_client.py and neo4j_client.py.
- Customize the design_template.json file according to your graph database schema.
- Run data_preparation.py to prepare the data for query generation.
- Execute query_generation.py to generate and evaluate queries against your Neo4j database.