/LLM_News

LOLA_ LLM-Assisted Online Learning Algorithm for Content Experiments

Primary LanguageJupyter Notebook

Code for paper LOLA: LLM-Assisted Online Learning Algorithm for Content Experiments.

Python Environment

We recommend using conda and pip to manage the environment. To set up the environment:

conda create --name lola
conda install pip
pip install datasets
pip install peft
pip install evaluate
pip install transformers -U
pip install -U scikit-learn
pip install -U matplotlib
pip install progressbar2
pip install openai

# to download the Llama-3 model (only needed for fine-tuning Llama-3), register on huggingface for access to the model and then run the following command
pip install -U "huggingface_hub[cli]"
huggingface-cli login
# type in your huggingface credentials

Dataset and Code

The original dataset we used is https://osf.io/jd64p/.

The pre-processed dataset can be downloaded from Kaggle, or use the kaggle CLI command: kaggle datasets download -d shuffleofficial/lola-llm-assisted-online-learning-algorithm

  • For data processing

    • Code Path Upworthy Data Processing.ipynb
    • Running this code will generate a csv file named winner-all.csv
    • Data used: upworthy-archive-holdout-packages-03.12.2020.csv, upworthy-archive-exploratory-packages-03.12.2020.csv and upworthy-archive-confirmatory-packages-03.12.2020.csv (these data are downloaded from https://osf.io/jd64p/)

  • For Prompt Engineering Method

    • Code Path Pure LLM Approaches/Pure LLM - Prompt/Prompt-based Approaches.ipynb
    • Data used winner-all.csv

  • For Classification using OpenAI and Word2Vec Embedding

    • Code Path Pure LLM Approaches/Pure LLM - Embedding/Embedding (OpenAI&Word2Vec) Classification.ipynb
    • Data used: selected_pairs_df_005_256.csv and selected_pairs_df_005_3072.csv

  • For Predicting CTR using OpenAI Embedding

    • Code Path LOLA/LOLA - Regret Minimize/LOLA_regret_minimize.ipynb
    • Data used: all_test_headline_embed_3072.csv

  • Survey Results

    • Code and data path Survey