/SpatialEval

Primary LanguagePython

SpatialEval

Setup Environments

This repository has been tested with Python version 3.10.13.

Install requirements with the command:

pip install -r requirements.txt

Task 1. Draw some points in 2d grid

  1. Synthesize data 
    python 2d_grid_point/2d_grid.py <n_samples> <points_in_a_grid> <grid_size>

The data will be saved in data/2d_grid_samples<n_samples>_points<points_in_a_grid>_size<grid_size>.json.

  1. Evaluate and save results

    a. Set your OPENAI_API_KEY in utils/chat_models.py

    b. Run the following command

    python 2d_grid_point/eval_2d_grid.py --path=<path_to_data> --model=<model_name>

    We can get an accuracy after the evaluation.

Task 2. 2D Path Planning

  1. Synthesize data

    cd path_planning
python path_planning/RandomObstacles.py

  2. Evaluate and save results

    a. Set your OPENAI_API_KEY in utils/chat_models.py

    b. Run the following command

    python path_planning/get_gpt_answer.py  
python path_planning/format_gpt4_output.py  
python path_planning/evaluate_gpt4_answer.py

    We can get an accuracy after the evaluation.