RL-based bidding strategy

This is a repository of the experiment code supporting the paper Real-time Bidding Strategy in Display Advertising: An Empirical Analysis.

Chinese README: README-zh

Getting Started

Data

We use benchmark dataset iPinYou. To download and formalize the iPinYou RTB data, please refers to make-ipinyou-data.

After the formalization, copy train.log.txt and text.log.txt of each campaign (e.g. 1458) to RLBid_EA/data/ipinyou/1458 for further use.

CTR Prediction

Before training the bidding strategy, you first need to predict the click-through rate of each ad impression. We provide pre-trained FM model parameters for 4 datasets in RLBid_EA/ctr/models. You can use them in conjunction with RLBid_EA/ctr/generate_pctr.py.

You are of course able to train a CTR predictor. RLBid_EA/ctr/model.py implemented 9 classical click-through prediction models.

Model	Paper	Link
LR	Predicting clicks: estimating the click-through rate for new ads	[paper]
FM	Factorization machines	[paper]
FFM	Field-aware factorization machines for CTR prediction	[paper]
W&D	Wide & deep learning for recommender systems	[paper]
PNN	Product-based neural networks for user response prediction	[paper]
DeepFM	DeepFM: a factorization-machine based neural network for CTR prediction	[paper]
FNN	Deep learning over multi-field categorical data	[paper]
DCN	Deep & cross network for ad click predictions	[paper]
AFM	Attentional factorization machines: Learning the weight of feature interactions via attention networks	[paper]

For details please refer to README .

Bidding Strategy

This repository implements 2 static bidding strategies and 3 dynamic bidding strategies based on reinforcement learning.

Model	Paper	Link
LIN	Bid optimizing and inventory scoring in targeted online advertising	[paper]
ORTB	Optimal real-time bidding for display advertising	[paper] [code]
RLB	Real-time bidding by reinforcement learning in display advertising	[paper] [code]
DRLB	Budget constrained bidding by model-free reinforcement learning in display advertising	[paper]
FAB	A dynamic bidding strategy based on model-free reinforcement learning in display advertising	[paper] [code]

For details please refer to the README in each folder.

License

This project is licensed under the Apache License 2.0.

Acknowledgments

@JiaXingBinggan contributed the CTR prediction model code for this repository.