The course introduces the use of open-source large language models (LLMs) from the Hugging Face ecosystem for research in the behavioral and social sciences.
By Dirk Wulff and Zak Hussain
09:15 AM - 09:45 AM: Welcome & Intro
09:45 AM - 10:45 AM: Talk: Intro to LLMs
10:45 AM - 11:00 AM: Break
11:00 AM - 12:00 PM: Discussion: Find applications in small groups
12:00 PM - 01:00 PM: Lunch
01:00 PM - 02:00 PM: Talk: A gentle intro to Hugging Face and Python
02:00 PM - 02:15 PM: Setup Colab
02:15 PM - 02:45 PM: Exercise: Running pipelines
02:45 PM - 03:15 PM: Walkthrough
09:15 AM - 09:45 AM: Recap quiz
09:45 AM - 10:45 AM: Talk: Intro to transformers & embeddings
10:45 AM - 11:00 AM: Break
11:00 AM - 12:00 PM: Talk: Intro to transformers & embeddings (continued)
12:00 PM - 01:00 PM: Lunch
01:00 PM - 02:00 PM: Exercise: Clarifying personality psychology
02:00 PM - 02:30 PM: Walkthrough
02:30 PM - 03:15 PM: Discussion: Find applications in small groups
09:15 AM - 09:45 AM: Recap quiz
09:45 AM - 10:45 AM: Intro to classification and regression
10:45 AM - 11:00 AM: Break
11:00 AM - 12:00 PM: Exercise: Classifying media bias w/ zero & few-shot
12:00 PM - 01:00 PM: Lunch
01:00 PM - 02:00 PM: Exercise: Classifying media bias w/ fine-tuning
02:00 PM - 02:30 PM: Walkthrough
02:30 PM - 03:15 PM: Discussion: Find applications in small groups
09:15 AM - 09:45 AM: Recap quiz
09:45 AM - 10:45 AM: Intro to text generation & synthetic participants
10:45 AM - 11:00 AM: Break
11:00 AM - 12:00 PM: Exercise: Numeracy
12:00 PM - 01:00 PM: Lunch
01:00 PM - 02:00 PM: Exercise: Steering vaccine hesitancy response
02:00 PM - 02:30 PM: Walkthrough
02:30 PM - 03:15 PM: Discussion: Find applications in small groups
09:15 AM - 09:45 AM: Recap quiz
09:45 AM - 10:45 AM: Talk: Intro to qualitative data analysis
10:45 AM - 11:00 AM: Break
11:00 AM - 12:00 PM: Exercise: Article and interview Q&A
12:00 PM - 01:00 PM: Lunch
01:00 PM - 03:00 PM: Project pitches
03:00 PM - 03:15 PM: Wrap up
The course grade will be determined based on the quality of a project pitch at the end of the course and a two-page research paper submitted after the course. The paper communicates an analysis applying large language models to a personal research question, including all parts of a traditional research paper (introduction, method, results, and discussion). The research paper can be based on the examples during the course. The research paper must be submitted via mail (wulff at mpib-berlin.mpg.de) by June 28th. To enroll for the research paper send us the slides of your project sketch presentation via mail immediately after the conclusion of the course.
title={A tutorial on open-source large language models for behavioral science},
url={osf.io/preprints/psyarxiv/f7stn},
publisher={PsyArXiv},
author={Hussain, Zak and Binz, Marcel and Mata, Rui and Wulff, Dirk U},
year={2023},
month={Dec},
doi={https://doi.org/10.31234/osf.io/f7stn}
}
Hugging face documentation
Hugging face book
But what is a GPT (3Blue1Brown)
- If you do not have a Google account, you will need to create one (this can be deleted after the workshop).
- Navigate to Google Drive (https://drive.google.com/).
- In the top-left, click New > More > Colaboratory. If you do not see Colaboratory, you may need to click "Connect more apps", search for 'Colaboratory', and install it. Then click New > More > Colaboratory.
- Copy the following code snipped into the first cell of the notebook. Run it (
shift + enteror click ► button) to mount your Google Drive to the Colab environment. A pop-up will ask you to connect; click through the steps to connect your Google Drive to Colab (you will have to do this every time you open a new notebook).
from google.colab import drive
drive.mount("/content/drive")
- Create a second cell in your notebook using the "+ Code" button that appears when you hover your cursor right under the first cell. Copy and run the following code snippet in the second cell of your notebook to clone the GitHub repository to your Google Drive :
%cd /content/drive/MyDrive
!git clone https://github.com/Zak-Hussain/LLM4BeSci_GSERM2024.git
- Go back to your Google Drive and navigate to the folder "LLM4SocBeSci". You should see the directories
day_1,day_2,day_3,day_4, andday_5containing the relevant notebooks (.ipynb files) and data (it may take a couple of minutes for the files to appear) for the exercises of each day. - Open the folder
day_1and then theday_1.ipynbnotebook. A new Colab window will open. - Run the first cell of the notebook to install the required packages. This may take a few minutes and ask for you to give permission to access your Google Drive.
You are now ready to start the exercises!