FastXML / PFastXML / PFastreXML - Fast and Accurate Tree Extreme Multi-label Classifier

This is a fast implementation of FastXML, PFastXML, and PFastreXML based on the following papers:

"FastXML: A Fast, Accurate and Stable Tree-classifier for eXtreme Multi-label Learning" Paper
"Extreme Multi-label Loss Functions for Recommendation, Tagging, Ranking & Other Missing Label Application" Paper
"DiSMEC - Distributed Sparse Machines for Extreme Multi-label Classification" Paper Code

DiSMEC makes it's appearance via an L2 penalty rather than an L1 which, when set with a high alpha and sparsity eps of 0.01-0.05, also can produce sparse linear classifiers.

It's implemented in the quasi-familiar scikit-learn clf format.

Release Notes

2.0

Version 2.0 is not backward compatible with 1.x
User model.save(path) to save models instead of cPickle
Rewrites data storage layer
Uses 50% the memory, loads 30% faster, and is 40% faster to inference

Binary

This repo provides a simple script along with the library, fxml.py, which allows easy train / testing of simple datasets.

It takes two formats: a simple JSON format and the standard extreme multi label dataset format.

Standard Benchmark Datasets

As an example, to train a standalone classifier against the Delicious-200K dataset:

fxml.py delicious.model deliciousLarge_train.txt --standard-dataset --verbose train --iters 5 --trees 20 --label-weight propensity --alpha 1e-4 --leaf-classifiers --no-remap-labels

To test:

fxml.py delicious.model deliciousLarge_test.txt --standard-dataset inference

JSON File

As fxml.py is intended as an easy to understand example for setting up a FastXML classifier, the JSON format is very simple. It is newline delimited format.

train.json:

{"title": "red dresses", "tags": ["clothing", "women", "dresses"]}
{"title": "yellow dresses for sweet 16", "tags": ["yellow", "summer dresses", "occasionwear"]}
...

It can then be trained:

fxml.py my_json.model train.json --verbose train --iters 5 --trees 20 --label-weight propensity --alpha 1e-4 --leaf-classifiers

Not the omission of the flags "--standard-dataset" and "--no-remap-labels". Since the tags/classes provided are strings, fxml.py will remap them to an integer label space for training. During inference, it will map the label index back

Simple Python Usage

from fastxml import Trainer, Inferencer

X = [Sparse or numpy arrays]
y = [[1, 3]] # Currently requires list[list[int]]

trainer = Trainer(n_trees=32, n_jobs=-1)

trainer.fit(X, y)

trainer.save(path)

clf = Inferencer(path)

clf.predict(X)
# or
clf.predict(X, fmt='dict')

#############
# PFastXML
#############

from fastxml.weights import propensity

weights = propensity(y)
trainer.fit(X, y, weights)

###############
# PFastreXML
###############
trainer = Trainer(n_trees=32, n_jobs=-1, leaf_classifiers=True)
trainer.fit(X, y, weights)

TODO

Run all the standard benchmark datasets against it.
Refactor. Most of the effort has been spent on speed and it needs to be cleaned up.

gaoyz0625/fastxml