Unity project with the results of the training of an artificial intelligence (AI) with the objective of creating simple humanoid procedural animations of defensive combat movements with a sword against an enemy attack using inverse kinematics (IK) and ML-Agents.
- Unity 2020.1.13f1
- ML-Agents Release 12
Useful links to learn some aspects of this project:
- Inverse kinematics
- Machine learning and reinforcement learning
- Deep learning and deep reinforcement learning
- ML-Agents
The demo consists of a list of attack animations that can be selected from the drop-down menu that appears on the screen. When one is selected, it is reproduced and the samurai AI performs the procedural defense animation.
VideoDemo.mp4
Download the following ZIP (build) and execute "TFGProject.exe".
-
Attacker: A humanoid (a knight with a shield and sword) that has premade attack animations that hit the defender.
-
Defender (AI): A humanoid (a samurai with his katana) that defense himself (moving the katana and blocking the enemy blow) using the AI.
The animations are created by two components:
- An AI that applies a force to the katana in order to move it and make it collide with the attacker's sword at both midpoints of its blades.
- IK applied to the defender's arms so that they follow the movement of the sword created by the AI.
This artificial intelligence is trained by deep reinforcement learning, an artificial neural network (ANN) with a reinforcement learning algorithm, both implemented by the ML-Agents project.
The agent in this training environment can move and rotate the sword as if the samurai was moving it with his arms. In more technical terms, the actions that the agent can perform are 5 discrete actions ("move" in the X, Y, and Z axes, and "rotate" in the X and Z axes) with three options each (apply negative value, apply positive value, or not apply any new value).
As for his observations, from the environment, the agent will know the following.
3D Position and rotation:
- Both swords
- Full body of the samurai divided in Head, Spine, Left and Right Arm, Hip, and Left and Right Leg
Velocity:
- Both swords
The next image shows how it is represented in Unity.
All the configuration data (reinforce algorithm used, ANN settings and hyperparameters, etc) is listed in the next image.
behaviors:
DefendSamurai:
trainer_type: ppo
hyperparameters:
batch_size: 512
buffer_size: 12000
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: linear
network_settings:
normalize: true
hidden_units: 512
num_layers: 3
vis_encode_type: simple
memory: null
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
init_path: null
keep_checkpoints: 5
checkpoint_interval: 500000
max_steps: 30000000
time_horizon: 1000
summary_freq: 50000See Training Configuration File in order to understand the previous image.
- Unity 2020.1 or later
- Python 3.6.1 or later
- ML-Agents release 12 or later *
- Download the project and add it to Unity Hub.
Third party resources: