/BayesUCB-river

My final project for the course Data Stream Mining with the River library

Primary LanguagePython

This is the "Data Stream Mining" project of Yuyan Zhao and Bérénice Jaulmes.

Documentation

  • bayes_ucb.py

It implements the BayesUCB algorithm[1] for the multi-armed bandit problem with the River library. We chose to use a Beta distribution to compute the posterior distribution, and use the p-th quantile as the upper confidence bound (UCB) for each arm. The arm with the highest UCB is then pulled. And the posterior distribution for the pulled arm is updated.

  • test.py

It evaluates the performance of the BayesUCB policy defined, and compares it with the existing bandit algorithms in River. The result is shown in the figure below: alt text

  • /previous_version

Our current code has been reorganized by Max via Pull requests to the River. The previous versions are also available at /previous_version

Reference

[1] Kaufmann, Emilie, Olivier Cappé, and Aurélien Garivier. "On Bayesian upper confidence bounds for bandit problems." Artificial intelligence and statistics. PMLR, 2012.