ToR[e]cSys

ToR[e]cSys is a PyTorch Framework to implement recommendation system algorithms, including but not limited to click-through-rate (CTR) prediction, learning-to-ranking (LTR), and Matrix/Tensor Embedding. The project objective is to develop a ecosystem to experiment, share, reproduce, and deploy in real world in a smooth and easy way (Hope it can be done).

Installation
Implemented Models
Documentation
Getting Started
Examples
Authors
License

Installation

TBU

Documentation

The complete documentation for ToR[e]cSys is available via ReadTheDocs website.
Thank you for ReadTheDocs! You are the best!

Implemented Models

1. Subsampling

Model Name	Research Paper	Year
Word2Vec	Omer Levy et al, 2015. Improving Distributional Similarity with Lessons Learned from Word Embeddings	2015

2. Negative Sampling

Model Name	Research Paper	Year
TBU

3. Click through Rate (CTR) Model

Model Name	Research Paper	Year
Logistic Regression	/	/
Factorization Machine	Steffen Rendle, 2010. Factorization Machine	2010
Factorization Machine Support Neural Network	Weinan Zhang et al, 2016. Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction	2016
Field-Aware Factorization Machine	Yuchin Juan et al, 2016. Field-aware Factorization Machines for CTR Prediction	2016
Product Neural Network	Yanru QU et al, 2016. Product-based Neural Networks for User Response Prediction	2016
Attentional Factorization Machine	Jun Xiao et al, 2017. Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks	2017
Deep and Cross Network	Ruoxi Wang et al, 2017. Deep & Cross Network for Ad Click Predictions	2017
Deep Factorization Machine	Huifeng Guo et al, 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction	2017
Neural Collaborative Filtering	Xiangnan He et al, 2017. Neural Collaborative Filtering	2017
Neural Factorization Machine	Xiangnan He et al, 2017. Neural Factorization Machines for Sparse Predictive Analytics	2017
eXtreme Deep Factorization Machine	Jianxun Lian et al, 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems	2018
Deep Field-Aware Factorization Machine	Junlin Zhang et al, 2019. FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine	2019
Deep Matching Correlation Prediction	Wentao Ouyang et al, 2019. Representation Learning-Assisted Click-Through Rate Prediction	2019
Deep Session Interest Network	Yufei Feng et al, 2019. Deep Session Interest Network for Click-Through Rate Prediction	2019
Elaborated Entire Space Supervised Multi Task Model	Hong Wen et al, 2019. Conversion Rate Prediction via Post-Click Behaviour Modeling	2019
Entire Space Multi Task Model	Xiao Ma et al, 2019. Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate	2019
Field Attentive Deep Field Aware Factorization Machine	Junlin Zhang et al, 2019. FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine	2019
Positon-bias aware learning framework	Huifeng Guo et al, 2019. PAL: a position-bias aware learning framework for CTR prediction in live recommender systems	2019

4. Embedding Model

Model Name	Research Paper	Year
Matrix Factorization	/	/
Starspace	Ledell Wu et al, 2017 StarSpace: Embed All The Things!	2017

5. Learning-to-Rank (LTR) Model

Model Name	Research Paper	Year
Personalized Re-ranking Model	Changhua Pei et al, 2019. Personalized Re-ranking for Recommendation	2019

Getting Started

There are several ways using ToR[e]cSys to develop a Recommendation System. Before talking about them, we first need to discuss about components of ToR[e]cSys.

A model in ToR[e]cSys is constructed by two parts mainly: inputs and model, and they will be wrapped into a sequential module (torecsys.models.sequential) to be trained by Trainer (torecsys.trainer.Trainer). \

For inputs module (torecsys.inputs), it will handle most kinds of inputs in recommendation system, like categorical features, images, etc, with several kinds of methods, including token embedding, pre-trained image models, etc.

For models module (torecsys.models), it will implement some famous models in recommendation system, like Factorization Machine famiry. I hope I can make the library rich. To construct a model in the module, in addition to the modules implemented in PyTorch, I will also implement some layers in torecsys.layers which are called by models usually.

After the explanation of ToR[e]cSys, let's move on to the Getting Started. We can use ToR[e]cSys in the following ways:

Run by command-line (In development)

> torecsys build --inputs_config='{}' \
--model_config='{"method":"FM", "embed_size": 8, "num_fields": 2}' \
--regularizer_config='{"weight_decay": 0.1}' \
--criterion_config='{"method": "MSELoss"}' \
--optimizer_config='{"method": "SGD", "lr": "0.01"}' \
...

Run by class method

import torecsys as trs

# build trainer by class method
trainer = trs.trainer.Trainer() \
    .bind_objective("CTR") \
    .bind_inputs() \
    .build_model(method="FM", embed_size=8, num_fields=2) \
    .build_sequential() \
    .build_regularizer(weight_decay=0.1) \
    .build_criterion(method="MSELoss") \
    .build_optimizer(method="SGD", lr="0.01") \
    .build_loader(name="train", ...) \
    .build_loader(name="eval", ...) \
    .set_targets_name("labels") \
    .set_max_num_epochs(10) \
    .use_cuda()

# start to fit the model
trainer.fit()

Run like PyTorch Module

import torch
import torch.nn as nn
import torecsys as trs

# some codes here
inputs = trs.inputs.InputsWrapper(schema=schema)
model = trs.models.FactorizationMachineModel(embed_size=8, num_fields=2)

for i in range(epochs):
    optimizer.zero_grad()

    outputs = model(**inputs(batchs))
    loss = criterion(outputs, labels)

    loss.backward()
    optimizer.step()

(In development) You can anyone you like to train a Recommender System and serve it in the following ways:

Run by command-line
```
> torecsys serve --load_from='{}'
```

Run by class method

import torecsys as trs

serving = trs.serving.Model() \
    .load_from(filepath=filepath)
    .run()

Serve it yourself

from flask import Flask, request
import torecsys as trs

model = trs.serving.Model() \
    .load_from(filepath=filepath)

@app.route("/predict")
def predict():
    args = request.json
    inference = model.predict(args)
    return inference, 200

if __name__ == "__main__":
    app.run()

For further details, please refer to the example in repository or read the documentation. Hope you enjoy~

Examples

TBU

Sample Codes

TBU

Sample of Experiments

TBU

Authors

Jasper Li - Developer

License

ToR[e]cSys is MIT-style licensed, as found in the LICENSE file.

jqsl2012/torecsys