Reproducibility package for the article:

An introduction to the bootstrap: a versatile method to make inferences by using data-driven simulations Rousselet G.A., Pernet C.R., Wilcox R.R. Meta-Psychology

[OSF repository] [GitHub repository] [PsyArXiv Preprint]

The repository contains all of the R code used in the article. The code is best seen by running the RMarkdown notebooks, within RStudio.

Content

folder	description
`code`	R `.Rmd` files to run simulations and create figures
`notebooks`	pdf versions of the code, with embedded figures
`data`	simulation results needed to run the code
`figures`	all the figures used in the article, in pdf format (only available on the OSF version of the repo)
`functions`	extra R functions defined in text files
`docs`	Github html versions of the notebooks (only available on the GitHub version of the repo)

Notebooks

The notebooks contain code to reproduce the figures and analyses presented in the article. They also contain extra resources, figures and analyses.

Notebook	Description	Figures
pb	Description of the percentile bootstrap	Figure 1
pc	Percent correct example	Figure 2
sampdist	Illustrate bootstrap sampling distributions	Figures 3-4
coverage	Simulations of the coverage, width and power of one-sample confidence intervals	Figures 5, 7, 8
notrobust	On its own, the bootstrap does not guarantee robustness	Figure 6
2indgps	Compare 2 independent groups	Figure 9
compcorr	Comparison of correlation coefficients	Figures 10-11
2depgps	Illustrate hierarchical bootstrap sampling	Figure 12
ptb	Percentile-t bootstrap technique	Figures 13-14

Direct links to the PDF version of the figures on the OSF

R packages needed

If you want to run the code in RStudio, you will need to install a few packages.

To reproduce the figures only, you can install the required packages by typing this in the console:

install.packages(c("ggplot2", "tibble"))

Or you can navigate in the GUI to Tools > Install Packages...

To install rogme and facetscales, first you might need to install devtools:

install.packages("devtools")

then:

devtools::install_github("GRousselet/rogme")

devtools::install_github("zeehio/facetscales")

To reproduce the summary figures, you also need cowplot to combine panels:

install.packages("cowplot")

Finally, if you decide to run the simulations, you will need beepr to get a little auditory reward:

install.packages("beepr")

Additional R functions

Here we highlight a few R functions relevant to the tutorial. Most of them are listed in the RMarkdown notebooks, with example syntax. Each notebook will install the appropriate functions for you; otherwise, in the console you can type source(file.choose()) and select the relevant .txt file.

Robust estimation and hypothesis testing

To get all the statistical functions from Rand Wilcox, select the Rallfun-v40.txt file. See details on this webpage. The full description of the functions is available in the book Introduction to Robust Estimation and Hypothesis Testing. Here are some of the functions used or mentioned in the notebooks.

One-sample

Name	Description
`onesampb`	one-sample percentile bootstrap for any estimator
`sint`	parametric inference on the median
`trimci`	one-sample test on trimmed means
`trimpb`	percentile bootstrap inferences on trimmed means
`trimcibt`	bootstrap-t on trimmed means
`hdpb`	percentile bootstrap inferences on the Harrell-Davis quantile estimator

Two independent groups

Name	Description
`yuen`	t-test on trimmed means
`yuenbt`	bootstrap-t on trimmed means
`pb2gen`	percentile bootstrap to compare any estimators
`medpb2`	same as pb2gen but only to compare medians
`trimpb2`	same as pb2gen but only to compare trimmed means
`comvar2`	parametric test to compare variances

Two dependent groups

Name	Description
`yuend`	t-test on dependent trimmed means
`ydbt`	bootstrap-t on trimmed means
`comdvar`	parametric test of variances
`bootdpci`	percentile bootstrap using any estimator

Correlations

Name	Description
`pcorb`	percentile bootstrap confidence interval for Pearson's correlation
`corb`	percentile bootstrap confidence interval for any robust correlation
`wincor`	winsorised correlation
`pbcor`	percentage bend correlation
`mscor`	skipped correlations using Pearson's or Spearman's correlation

Compare independent correlations

Name	Description
`twopcor`	percentile bootstrap comparison of two independent Pearson's correlations
`twocor`	percentile bootstrap comparison of two independent robust correlations

Compare dependent correlations

Name	Description
`TWOpov`	compare two overlapping Pearson's correlations
`twoDcorR`	compare two overlapping robust correlations
`TWOpNOV`	compare two non-overlapping Pearson's correlations
`twoDNOV`	compare two non-overlapping robust correlations

Other custom functions

functions.txt and theme_gar.txt contain custom code to set some ggplot2 parameters and to compute a few things. Other custom functions are defined in the notebooks.