This is the PyTorch implementation of MAAB. The paper can be found here.
The code includes the experiments for the following two environments:
- two-agent bidding game
- offline dataset simulation
and the implementations of the following algorithms:
- CM-IL
- CO-IL
- MAAB (iql_vrl)
- MAAB-fix (iql_vfix)
- DQN-S (iql_ali_single)
This is a simplified bidding environment with only two agents bidding either in a competitive or cooperative manner.
For a competitive manner (CM-IL), run
python src/main.py --config=iql --env-config=auction with batch_size=32 env_args.coop=0 # CM-ILFor a cooperative manner (CO-IL), run
python src/main.py --config=iql --env-config=auction with batch_size=32 env_args.coop=100 # CO-ILMAAB is also provided in the two-agent bidding game:
python src/main.py --config=iql_vrl --env-config=auction with batch_size=32 env_args.coop=4 # MAAB (with bar agents)For MAAB in offline simulation, run
python src/main.py --config=iql_ali_vfix --env-config=auction_ali with batch_size=32 v_threshold=0.5 env_args.coop=4 Note that the dataset for the offline simulation is not provided due to data security concern.
The running results are stored in the results/tb_logs folder, in the tensorboard format. You can view the logs as well as the results by running
tensorboard --logdir results/tb_logs @inproceedings{wen2022maab,
title={A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising},
author={Wen, Chao and Xu, Miao and Zhang, Zhilin and Zheng, Zhenzhe and Wang, Yuhui and Liu, Xiangyu and Rong, Yu and Xie, Dong and Tan, Xiaoyang and Yu, Chuan and others},
booktitle={WSDM 2022},
year={2022}
}