/ssrm

Sequential Sample Ratio Mismatch(SRM) Test

Primary LanguagePythonApache License 2.0Apache-2.0

Build

ssrm-logo SSRM: A Sequential Sample Ratio Mismatch Test

A package for sequential testing of Sample Ratio Mismatch (SRM).

See the white paper.

See the blog post.

Contributors:

Installation

We recommend that you use an isolated virtual environment to install and run the code in this repo (See: virtualenv and pyenv)

  1. You can clone the repository from the command-line: 
    git clone git@github.com:optimizely/ssrm.git
    We highly recommend that you branch off of master and pull in upstream changes regularly.
  2. Install dependencies: Run make install.
    • If you wish to develop in the repo, run make install-dev. Also, see the contributing doc here

    Tip: have a look in the Makefile to learn more about what this, and other make recipes do!

  3. Run tests:
    • make check to run all checks.
    • make test to run unit tests.

Tutorials

We provide a tutorial notebook that walks through an example of running a Sequential SRM test here. Run jupyter lab, and open notebooks/introduction.ipynb.

Documentation

The latest reference documentation is here.

Contributing

See the contributing doc here.

Credits

First-party code (under ssrm_test) is copyright Optimizely, Inc. and contributors, licensed under Apache 2.0.

Additional Code

This software incorporates code from the following open source projects:

numpy https://numpy.org/index.html

scipy https://www.scipy.org/scipylib/index.html

toolz https://github.com/pytoolz/toolz

typing https://github.com/python/typing