/adanet

Fast and flexible AutoML with learning guarantees.

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

AdaNet

adanet_tangram_logo

Documentation Status PyPI version Travis codecov Gitter Downloads License

AdaNet is a lightweight TensorFlow-based framework for automatically learning high-quality models with minimal expert intervention. AdaNet builds on recent AutoML efforts to be fast and flexible while providing learning guarantees. Importantly, AdaNet provides a general framework for not only learning a neural network architecture, but also for learning to ensemble to obtain even better models.

This project is based on the AdaNet algorithm, presented in “AdaNet: Adaptive Structural Learning of Artificial Neural Networks” at ICML 2017, for learning the structure of a neural network as an ensemble of subnetworks.

AdaNet has the following goals:

  • Ease of use: Provide familiar APIs (e.g. Keras, Estimator) for training, evaluating, and serving models.
  • Speed: Scale with available compute and quickly produce high quality models.
  • Flexibility: Allow researchers and practitioners to extend AdaNet to novel subnetwork architectures, search spaces, and tasks.
  • Learning guarantees: Optimize an objective that offers theoretical learning guarantees.

The following animation shows AdaNet adaptively growing an ensemble of neural networks. At each iteration, it measures the ensemble loss for each candidate, and selects the best one to move onto the next iteration. At subsequent iterations, the blue subnetworks are frozen, and only yellow subnetworks are trained:

adanet_tangram_logo

AdaNet was first announced on the Google AI research blog: "Introducing AdaNet: Fast and Flexible AutoML with Learning Guarantees".

This is not an official Google product.

Features

AdaNet provides the following AutoML features:

Example

A simple example of learning to ensemble linear and neural network models:

import adanet
import tensorflow as tf

# Define the model head for computing loss and evaluation metrics.
head = MultiClassHead(n_classes=10)

# Feature columns define how to process examples.
feature_columns = ...

# Learn to ensemble linear and neural network models.
estimator = adanet.AutoEnsembleEstimator(
    head=head,
    candidate_pool={
        "linear":
            tf.estimator.LinearEstimator(
                head=head,
                feature_columns=feature_columns,
                optimizer=...),
        "dnn":
            tf.estimator.DNNEstimator(
                head=head,
                feature_columns=feature_columns,
                optimizer=...,
                hidden_units=[1000, 500, 100])},
    max_iteration_steps=50)

estimator.train(input_fn=train_input_fn, steps=100)
metrics = estimator.evaluate(input_fn=eval_input_fn)
predictions = estimator.predict(input_fn=predict_input_fn)

Getting Started

To get you started:

Requirements

Requires Python 3.6 or above.

adanet is built on TensorFlow 2.1. It depends on bug fixes and enhancements not present in TensorFlow releases prior to 2.1. You must install or upgrade your TensorFlow package to at least 2.1:

$ pip install "tensorflow==2.1"

Installing with Pip

You can use the pip package manager to install the official adanet package from PyPi:

$ pip install adanet

Installing from Source

To install from source first you'll need to install bazel following their installation instructions.

Next clone the adanet repository:

$ git clone https://github.com/tensorflow/adanet
$ cd adanet

From the adanet root directory run the tests:

$ bazel build -c opt //...
$ python3 -m nose

Once you have verified that the tests have passed, install adanet from source as a pip package .

You are now ready to experiment with adanet.

import adanet

Citing this Work

If you use this AdaNet library for academic research, you are encouraged to cite the following paper from the ICML 2019 AutoML Workshop:

@misc{weill2019adanet,
    title={AdaNet: A Scalable and Flexible Framework for Automatically Learning Ensembles},
    author={Charles Weill and Javier Gonzalvo and Vitaly Kuznetsov and Scott Yang and Scott Yak and Hanna Mazzawi and Eugen Hotaj and Ghassen Jerfel and Vladimir Macko and Ben Adlam and Mehryar Mohri and Corinna Cortes},
    year={2019},
    eprint={1905.00080},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

License

AdaNet is released under the Apache License 2.0.