This project is a reimplementation (with a few improvements) of Alphastar (Only Zerg vs Zerg) based on OpenDILab, which contains:
-
Play demo and test code (try and play with our agent!)
-
First version of pre-trained SL and RL agent
-
Training code of Supervised Learning (updated by 2022-01-31)
-
Training code of Reinforcement Learning with self-play and league (updated by 2022-01-31)
-
Agents fought with Harstem (YouTube) (New! updated by 2022-04-01)
-
More stronger pre-trained RL agents (WIP)
Please star us to help DI-star agents to grow up faster :)
For users unfamiliar with programming, follow instructions here
Environment requirement:
- Python: 3.6-3.8
- Download the retail version of StarCraftII: https://starcraft2.com
Note: There is no retail version on Linux, please follow the instruction here
-
Add SC2 installation path to environment variables
SC2PATH
(skip this if you use default installation path on MacOS or Windows, which isC:\Program Files (x86)\StarCraft II
or/Applications/StarCraft II
):-
On MacOS or Linux, input this in terminal:
export SC2PATH=<sc2/installation/path>
-
On Windows:
- Right-click the Computer icon and choose Properties, or in Windows Control Panel, choose System.
- Choose Advanced system settings.
- On the Advanced tab, click Environment Variables.
- Click New to create a new environment variable, set
SC2PATH
as the sc2 installation location. - After creating or modifying the environment variable, click Apply and then OK to have the change take effect.
-
git clone https://github.com/opendilab/DI-star.git
cd DI-star
pip install -e .
Pytorch Version 1.7.1 and CUDA is recommended, Follow instructions from pytorch official site
Note: GPU is neccessary for decent performance in realtime agent test, you can also use pytorch without cuda, but no performance guaranteed due to inference latency on cpu. Make sure you set SC2 at lowest picture quality before testing.
Double click file replay_4.10.0.SC2Replay
, StarCraftII version 4.10.0 will be automatically downloaded.
Note: We trained our models with versions from 4.8.2 to 4.9.3. Patch 5.0.9 has came out in March 15, 2022, Some changes have huge impact on performance, so we fix our version at 4.10.0 in evaluation.
python -m distar.bin.download_model --name rl_model
Note: Specify rl_model
or sl_model
after --name
to download reinforcement learning model or supervised model.
Model list:
sl_model
: training with human replays, skill is equal to diamond players.rl_model
: used as default, training with reinforcement learning, skill is equal to master or grandmaster.Abathur
: one of reinforcement learning models, likes playing mutalisk.Brakk
: one of reinforcement learning models, likes lingbane rush.Dehaka
: one of reinforcement learning models, likes playing roach ravager.
With the given model, we provide multiple tests with our agent.
python -m distar.bin.play
It runs 2 StarCraftII instances. First one is controlled by our RL agent. Human player can play on the second one with full screen like normal game.
Note:
- GPU and CUDA is required on default, add
--cpu
if you don't have these. - Download RL model first or specify other models (like supervised model) with argument
--model1 <model_name>
- In race cases, 2 StarCraftII instances may lose connection and agent won't issue any action. Please restart when this happens.
python -m distar.bin.play --game_type agent_vs_agent
It runs 2 StarCraftII instances both controlled by our RL Agent, specify other model path with argument --model1 <model_name> --model2 <model_name>
python -m distar.bin.play --game_type agent_vs_bot
RL agent plays against built-in elite bot.
StarCraftII client is required for replay decoding, follow instructions above.
python -m distar.bin.sl_train --data <path>
path could be either a directory with replays or a file includes a replay path at each line.
Optionally, separating replay decoding and model training could be more efficient, run the three scripts in different terminals:
python -m distar.bin.sl_train --type coordinator
python -m distar.bin.sl_train --type learner --remote
python -m distar.bin.sl_train --type replay_actor --data <path>
For distributed training:
python -m distar.bin.sl_train --init_method <init_method> --rank <rank> --world_size <world_size>
or
python -m distar.bin.sl_train --type coordinator
python -m distar.bin.sl_train --type learner --remote --init_method <init_method> --rank <rank> --world_size <world_size>
python -m distar.bin.sl_train --type replay_actor --data <path>
Here is an example of training on a machine with 4 GPUs in remote mode:
# Run the following scripts in different terminals (windows).
python -m distar.bin.sl_train --type coordinator
# Assume 4 GPUs are on the same machine.
# If your GPUs are on different machines, you need to configure the init_mehod's IP for each machine.
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 0 --world_size 4
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 1 --world_size 4
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 2 --world_size 4
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 3 --world_size 4
python -m distar.bin.sl_train --type replay_actor --data <path>
Reinforcement learning will use supervised model as initial model, please download it first, StarCraftII client is also required.
python -m disatr.bin.rl_train
python -m disatr.bin.rl_train --task selfplay
Four components are used for RL training, just like SL training, they can be executed through different process:
python -m distar.bin.rl_train --type league --task selfplay
python -m distar.bin.rl_train --type coordinator
python -m distar.bin.rl_train --type learner
python -m distar.bin.rl_train --type actor
Distributed training is also supported like SL training.
More configuration(e.g. batch size, learning rate, etc.) could be found at distar/bin/user_config.yaml
.
Training guide and baselines will be added soon.
Slack: link
Discord server: link
@misc{distar,
title={DI-star: An Open-sourse Reinforcement Learning Framework for StarCraftII},
author={DI-star Contributors},
publisher = {GitHub},
howpublished = {\url{https://github.com/opendilab/DI-star}},
year={2021},
}
DI-star released under the Apache 2.0 license.