- Hold-out Script - Experiment 1:
run_holdout.py
- Pre-sequential Script - Experiment 2:
run_pre_sequential.py
- Zero-shot and Fine-tuning with Lag-Llama:
run_finetune.py
Let's start by setting up your environment:
-
Create a Conda Environment:
conda create -n AML4CPU python=3.10.12 -y conda activate AML4CPU
-
Clone the Repository and Install Requirements:
git clone https://github.com/sebasmos/AML4CPU.git cd AML4CPU pip install -r requirements.txt
-
Install PyTorch and Other Dependencies:
pip install clean-fid numba numpy torch==2.0.0+cu118 torchvision --force-reinstall --extra-index-url https://download.pytorch.org/whl/cu118
Run the holdout evaluation script:
python run_holdout.py --output_file 'exp1' --output_folder Exp1 --num_seeds 20
Run the pre-sequential evaluation script:
python run_pre_sequential.py --output_file 'exp2' --eval --output_folder Exp2 --num_seeds 20
Test zero-shot over different context lengths (32, 64, 128, 256) with and without RoPE:
python run_finetune.py --output_file zs --output_folder zs --model_path ./models/lag_llama_models/lag-llama.ckpt --eval_multiple_zero_shot --max_epochs 50 --num_seeds 20
Finetune and test Lag-Llama over different context lengths (32, 64, 128, 256) with and without RoPE:
python run_finetune.py --output_file exp3_REAL_parallel --output_folder Exp3 --model_path ./models/lag_llama_models/lag-llama.ckpt --max_epochs 50 --num_seeds 20 --eval_multiple
This project is licensed under the MIT License. See LICENSE for details.