mingyus/explore-exploit

Data and code for "Sources of suboptimality in a minimalistic explore-exploit task"

Explore-Exploit

This repository accompanies the following paper:

Song M, Bnaya Z, Ma WJ (2019) Sources of suboptimality in a minimalistic explore-exploit task. Nature Human Behaviour 3, 361–368.

It includes human participants' data, the experiment code, and the analysis code for reproducing the results reported in the paper.

Data

Each .mat file contains the data from one participant. The variable data contains 1350 (for laboratory experiments) or 450 (for Mturk experiments) rows and 6 columns.

Each row is one choice, and the six columns are:

1. Trial number
2. Total number of days
3. Current day
4. Reaction time (in units of ms)
5. Choice/Action (1 represents exploration and 0 represents exploitation)
6. Reward

Experiment Code

Lab experiments (experiments 1 and 4)

How to run the experiments

The experimental codes for lab experiments run locally. To run the experiments, simply start from ExpX (lab)/templates/instructions/instruction1.html. It will go through the instructions, comprehension tests and the task (180 trials), followed by a post-task question on strategy.

How to obtain the data

The responses to the comprehension tests are automatically printed on the screen above the “Back” button after the participant’s responses. See variable trialdata in quiz1.js and quiz2.js for the meaning of the numbers.

After the participant completes the entire task, they will see a page starting with "Thanks for participating in the experiment!”. Press the “`” key and all the data for the task and the strategy question will be printed on the screen. The data is printed as an array in the sequence of the 180 trials (in total 1350 responses). Each response consists of 6 numbers (trial number, total number of days, current day, reaction time, choice - 1 for exploration and 0 for exploitation, reward). The array is followed by the answer to the strategy question.

Mturk experiments (experiments 2, 3 and 5)

How to run the experiments

The Mturk experiments are programmed with psiTurk to run on Mturk. Please refer to psiTurk documentation to see how to run the experiments both locally and on Mturk.

How to obtain the data

The data are recorded in the same format as lab experiments, but in a local database file called participants.db.

Analysis Code

Folders:

• data_extracted/: Extracted variables from the raw data. Used for plotting and analyses.

• fittingresult/: Model fitting results, including fitted parameters and negative log likelihood values for each participant and each model.

Scripts:

• Preprocessing script:

  • preprocessing_dataextract.m: extracts variables from the raw data

• Model fitting scripts:

  • modelfitting_Opt.m: script for fitting the Opt model
  • modelfitting_thresholdF.m: script for fitting the Num and Prop models
  • modelfitting_thresholdV.m: script for fitting the Num-V and Prop-V models
  • modelfitting_threshold39.m: script for fitting the Flexible-Threshold model
  • modelfitting_thresholdVrisk.m: script for fitting the Prop-V-risk model
  • modelfitting_thresholdV_halfdata.m: script for fitting the Prop-V model on the first half and the second half of the data separately for each participant
  • optimizefun_thresholdF.m: model function for the Num and Prop models
  • optimizefun_thresholdV.m: model function for the Num-V and Prop-V models
  • optimizefun_thresholdVrisk.m: model function for the Prop-V-risk model

• Plotting scripts:

  • FIG_optpolicy_colorplot.m: plots the optimal policy as a function of the highest reward so far and the number of days left
  • FIG_summarystats_simudata.m: script that generates summary statistics and the corresponding model fits
  • FIG_model_comparison.m: plots model comparison results (AICc and BIC)
  • FIG_thres39.m: plots the fitted thresholds of the Flexible-Threshold model
  • FIG_illustration_thresV.m: illustrations of the sequence-level variability

• Analysis scripts:

  • analysis_statistictests.m: statistical analyses
  • analysis_BMS.m: Bayesian model selection analysis

How to reproduce the analyses and figures in the paper

In general, use variable exp to set which experiment you want to analyze/plot, and use variable modelnum(modelarray) to set which model(s) you want to analyze/plot (see fun_modelnum2name.m for the correspondence between modelnum and model names)

Specifically, below are the codes to reproduce the main figures:

• Figure 1b: FIG_optpolicy_colorplot

• Figure 1c: FIG_summarystats_simudata(1,1,[],'colorplot')

• Figure 1d: FIG_summarystats_simudata(1,1,[],'psychometrics_rstar_tleft')

• Figure 1e: FIG_summarystats_simudata(1,1,[],'Teffect_subset')

• Figure 1f: FIG_summarystats_simudata(1,1,[],'numswitch')

• Figure 1g: FIG_summarystats_simudata(1,1,[],'avereward')

• Figure 2a: FIG_summarystats_simudata(2,1,1,'psychometrics_rstar_tleft')

• Figure 2b: FIG_summarystats_simudata(2,1,1,'Teffect_subset')

• Figure 2c: FIG_summarystats_simudata(2,1,1,'numswitch')

• Figure 3a: FIG_summarystats_simudata(2,1,2,'psychometrics_rstar_tleft')

• Figure 3b: FIG_summarystats_simudata(2,1,2,'Teffect_subset')

• Figure 3c: FIG_summarystats_simudata(2,1,2,'numswitch')

• Figure 3e: FIG_summarystats_simudata(2,1,3,'psychometrics_rstar_tleft')

• Figure 3f: FIG_summarystats_simudata(2,1,3,'Teffect_subset')

• Figure 3g: FIG_summarystats_simudata(2,1,3,'numswitch')

• Figures 3g & 3h: FIG_thres39

• Figures 4a & 4b: FIG_illustration_thresV (there is stochasticity in the code, so you won't get the exact same figures every time)

• Figure 4c: FIG_model_comparison (first section of the script)

• Figure 4d: FIG_summarystats_simudata(2,1,5,'psychometrics_rstar_tleft')

• Figure 4e: FIG_summarystats_simudata(2,1,5,'Teffect_subset')

• Figure 4f: FIG_summarystats_simudata(2,1,5,'numswitch')

Note that FIG_summarystats_simudata.m generates model fits by simulating the models using fitted parameters for each participant. For fast reproduction of the model fits figures (Figures 2, 3a-f, 4d-f), we recommend that you first run the following scripts to simulate and save all the data needed for plotting, and then run the scripts provided above by adding two extra input arguments Nrep(the size of the simulated data; we used 100 in the paper) and ifLoadSimu(set it to 1).

Scripts to simulate and save data in advance:

% set the size of the simulated data
Nrep = 100;  % 100 is what we used in the paper
% create the folder to save the simulated data
if exist('simulation_subj','dir')~=7
    mkdir('simulation_subj');
end
% simulate and save the data
[subarray,obsnum] = fun_exp2subarray2obsnum(exp);
for subnum = subarray
    for irep = 1:Nrep
        [simudata{irep},~] = simulate_data(modelnum,p_min,obsnum/39);
        save(['simulation_subj/' fun_modelnum2name(modelnum) '_sub' num2str(subnum) '_simusize' num2str(obsnum/39) '_Nrep' num2str(Nrep) '.mat']);
    end
end

For example, to reproduce Figure 2a, now run:

FIG_summarystats_simudata(2,1,1,'psychometrics_rstar_tleft',Nrep,1)