Source code to reproduce experiments from the article Practitioner’s Guide to Statistical Tests
To play with statistical tests use Sandbox.ipynb notebook. You can change the following data generation parameters:
success_rate = 0.02
uplift = 0.2
N = 5000 # Number of users in groups
NN = 2000 # Number of runs
beta = 1000
skew = 1
Read the article for the details.
To reproduce gifs and plots from the article use assets.ipynb notebook.