/abtester

Theory and implementatuon of various A/B experiments

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

abtester

Theory and implementatuon of various A/B experiments

Currently
Added examples for:

  • Standard T-test
  • Mann-Whitney U-test
  • Bootstrapped testing

Added theory for:

  • Standard tests
  • CUPED

Upcoming: A/B tests speed up

  • Stratification
  • Time dependency (A/B on WoW Changes, not raw values)
  • Linearization
  • CUPED
  • Bucketing

Interesting articles:

  • How to check A/B splitting systems, article
  • A/B at Avito
  • A/B framework at Airbnb
  • Presentation from Yandex
  • Technique for multiple A/B tests at Google
  • Comparison of various test statistics power from VK