bantmen/banditml

A lightweight & open source framework for personalization https://www.banditml.com

What's Bandit ML?

Bandit ML is a lightweight decision making & personalization framework built by experts from Facebook's applied reinforcement learning team, Reagent. Bandit ML gives users access to state of the art contextual bandit and reinforcement learning algorithms through a simple javascript widget.

This repo holds the open source machine learning code that powers banditml.com. Specifically, this repo contains:

• Feature engineering & preprocessing
• Model implementations
• Model training workflows

Supported models

Models supported:

• Contextual Bandits (small datasets)
  • Linear bandit w/ ε-greedy exploration
  • Random forest bandit w/ ε-greedy exploration
  • Gradient boosted decision tree bandit w/ ε-greedy exploration
• Contextual Bandits (medium datasets)
  • Neural bandit with ε-greedy exploration
  • Neural bandit with UCB-based exploration (via. dropout exploration)
  • Neural bandit with UCB-based exploration (via. mixture density networks)
• Reinforcement Learning (large datasets)
  • Deep Q-learning with ε-greedy exploration
  • Quantile regression DQN with UCB-based exploration
  • Soft Actor-Critic

4 feature types supported:

• Numeric: standard floating point features
  • e.g. {totalCartValue: 39.99}
• Categorical: low-cardinality discrete features
  • e.g. {currentlyViewingCategory: "men's jeans"}
• ID list: high-cardinality discrete features
  • e.g. {productsInCart: ["productId022", "productId109"...]}
  • Handled via. learned embedding tables
• "Dense" ID list: high-cardinality discrete features, manually mapped to dense feature vectors
  • e.g {productId022: [0.5, 1.3, ...], productId109: [1.9, 0.1, ...], ...}

Docs

If you just want to train a model for free and do everything else yourself these are the docs for you:

Do it yourself quick start

Alternatively, the hosted solution offers an end-to-end service for integrating contextual bandit and reinforcement learning algorithms into your application. Specifically, the end-to-end service offers:

• A UI to create, manage, and view bandit experiments
• A lightweight javascript client library to log data, get decisions, and provide feedback about decisions
• A UI to monitor, train, and evaluate bandit models
• Model understanding & insight tools

The docs to the hosted solution can be found here.

License

GNU General Public License v3.0 or later

See COPYING to see the full text.