/NA2Q

[ICML'23] Official PyTorch Implementation of NA2Q, and a comprehensive benchmark based on pymarl

Primary LanguagePython

NA2Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning

Code for NA2Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning accepted by ICML 2023. NA2Q is implemented in PyTorch and tested on challenging tasks LBF and SMAC as benchmarks, which is based on PyMARL. [paper] [code]

Python MARL framework

This PyMARL includes baselines of the following algorithms:

Thanks to all the original authors!

Installation instructions

Build the Dockerfile using

cd docker
bash build.sh

Set up StarCraft II and SMAC:

bash install_sc2.sh

This will download SC2 into the 3rdparty folder and copy the maps necessary to run over.

Requirements

  • Python 3.6+
  • pip packages listed in requirements.txt

The requirements.txt file can be used to install the necessary packages into a virtual environment (not recommended).

Run an experiment

# demo sc2
python src/main.py --config=qnam --env-config=sc2 with env_args.map_name=8m_vs_9m gpu_id=0 t_max=2010000 epsilon_anneal_time=50000 seed=1
# demo foraging
nohup python src/main.py --config=qnam --env-config=foraging with env_args.map_name=lbf-4-2 use_cuda=False seed=1 &

The config files act as defaults for an algorithm or environment.

They are all located in src/config. --config refers to the config files in src/config/algs --env-config refers to the config files in src/config/envs

All results will be stored in the Results folder.

Citing NA2Q

If you find this repository useful for your research, please cite it in BibTeX format:

@inproceedings{liu2023na2q,
  title = 	 {{NA$^2$Q}: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning},
  author =       {Liu, Zichuan and Zhu, Yuanyang and Chen, Chunlin},
  booktitle = 	 {Proceedings of the 40th International Conference on Machine Learning},
  pages = 	 {22539--22558},
  year = 	 {2023},
  volume = 	 {202},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {23--29 Jul},
  publisher =    {PMLR},
  url = 	 {https://proceedings.mlr.press/v202/liu23be.html},
}

In case of any questions, bugs, suggestions or improvements, please feel free to drop me or open an issue.

If you need the experimental results data from our paper, please contact me at zichuanliu@smail.nju.edu.cn and I'll be happy to share them!