lsm/banditml

A lightweight & open source framework for personalization https://www.banditml.com

A lightweight & open source framework for personalization

What's Bandit ML?

Bandit ML is a lightweight decision making & personalization framework built by experts from Facebook's applied reinforcement learning team, Reagent. Bandit ML gives users access to state of the art contextual bandit and reinforcement learning algorithms through a simple javascript widget.

This repo holds the open source machine learning code that powers banditml.com. Specifically, this repo contains:

• Feature engineering & preprocessing
• Model implementations
• Model training workflows

Supported models

Models supported:

• Gradient boosted decision tree contextual bandit with ε-greedy exploration (for small data problems)
• Neural contextual bandit with ε-greedy exploration
• Neural contextual bandit with UCB-based exploration (via. dropout exploration)
• Neural contextual bandit with UCB-based exploration (via. mixture density networks)
• Deep Q-learning with ε-greedy exploration
• Quantile regression DQN with ε-greedy exploration
• Soft Actor-Critic

4 feature types supported:

• Numeric: standard floating point features
  • e.g. {totalCartValue: 39.99}
• Categorical: low-cardinality discrete features
  • e.g. {currentlyViewingCategory: "men's jeans"}
• ID list: high-cardinality discrete features
  • e.g. {productsInCart: ["productId022", "productId109"...]}
  • Handled via. learned embedding tables
• "Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
  • e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

If you just want to train a model for free and do everything else yourself these are the docs for you:

Do it yourself quick start

Alternatively, the hosted solution offers an end-to-end service for integrating contextual bandit and reinforcement learning algorithms into your application. Specifically, the end-to-end service offers:

• A UI to create, manage, and view bandit experiments
• A lightweight javascript client library to log data, get decisions, and provide feedback about decisions
• A UI to monitor, train, and evaluate bandit models
• Model understanding & insight tools

The docs to the hosted solution can be found here.

License

GNU General Public License v3.0 or later

See COPYING to see the full text.