主要是一些MARL算法的pytorch实现,目前包括: VDN, QMIX, Weighted QMIX(CW-QMIX, OW-QMIX)
该项目基于 https://github.com/starry-sky6688/StarCraft 改进得到,简化了模块与算法流程,改进可视化,方便建立自己的算法库。
网络参数设置与“The StarCraft Multi-Agent Challenge”(SMAC)中保持一致。
目前在StarCraft II上进行测试,但可以方便地迁移到任意封装好的环境使用。
相关资料:Pymarl
- QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
- Value-Decomposition Networks For Cooperative Multi-Agent Learning
- Counterfactual Multi-Agent Policy Gradients
- QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
- Learning Multiagent Communication with Backpropagation
- From Few to More: Large-scale Dynamic Multiagent Curriculum Learning
- Multi-Agent Game Abstraction via Graph Attention Neural Network
- MAVEN: Multi-Agent Variational Exploration
- Weighted QMIX: Expanding Monotonic Value Function Factorisation
- The StarCraft Multi-Agent Challenge
- Python >= 3.6
- Pytorch >= 1.2
- SMAC
- Seaborn >= 0.9
对于SMAC,这里简单介绍一下linux下的安装,Windows等系统可以查看他们的仓库。
-
通过下列命令安装SMAC
pip install git+https://github.com/oxwhirl/smac.git
-
安装StarCraft II,这里给出 4.6.2 的下载链接,因为SMAC用的就是这个,并且他说不同版本之间不能比较,其余版本可以查看暴雪的仓库,解压时需要密码
iagreetotheeula
。解压后文件默认路径为~/StarCraftII/
,如果放在别的路径,需要更改环境变量SC2PATH
-
下载SMAC MAPS,解压后将文件夹直接放在
$SC2PATH/Maps
下即可 -
运行
python -m smac.bin.map_list
测试地图是否放置成功,运行python -m smac.examples.random_agents
测试安装是否成功。如果是centos的话可能会出现因为缺少对应版本依赖/home/user/SMAC/StarCraftII/Versions/Base75689/SC2_x64: /lib64/libc.so.6: version 'GLIBC_2.18' not found (required by /home/user/SMAC/StarCraftII/Libs/libstdc++.so.6)
而导致pysc2.lib.remote_controller.ConnectError: Failed to connect to the SC2 websocket. Is it up?
,这时候就要根据情况安装依赖或者使用docker了。
- Qatten
- Other SOTA MARL algorithms
- Update results on other maps
$ python main.py --map=3m --alg=qmix
或者直接pycharm打开项目,run main.py即可,也可以使用run.sh复现QMIX、CW-QMIX、OW-QMIX在5m_vs_6m上的实验。
SMAC的各种地图描述在这里:https://github.com/oxwhirl/smac/blob/master/docs/smac.md
暂时只贴一部分,因为我目前主要实现值分解的算法。所有地图的环境设置均与SMAC相同,难度为7(VeryHard)
![]() |
![]() |
---|---|
VDN | QMIX |
![]() |
![]() |
![]() |
---|---|---|
QMIX | CW-QMIX | OW-QMIX |
If you want to see the replay, make sure the replay_dir
is an absolute path, which can be set in ./common/arguments.py
. Then the replays of each evaluation will be saved, you can find them in your path.