Project will continue once I complete CS50ai.

TODO:

Multi-Layer Neural Network Support (more than a single hidden network)
PPO Support (Unrealistic but will try!)

https://github.com/ryash072007/Godot-AI-Kit/blob/d1fab400e05bb33b6b00007e4790bfca6c76b23a/videos/Godot%20AI%20Kit.mp4

AI Algorithm for Godot 4

The goal of this project is to provide a variety of AI Algorithms in Godot 4 natively using GDscript.

Simple Neural Network and Neural Net Plugin for Godot 4

This part of the plugin allows you to create a Multi Layer Neural Network and also provides a NeuralNet by which you can easily automatically train the network (which can be found under Node2D Section in the add node window).

This plugin is intended for creating AIs that can complete a game level.

Rules to be followed if using Neural Net

If using Neural Net, the identifier or name of the variable of the Neural Network used in your code has to be nn. Like this:

var nn: NeuralNetwork

This is because the Neural Net only works when the Neural Network is named as nn.

If using Neural Net, make sure you do not assign your Neural Network Variable nn anything. All you are supposed to do is declare it like this:

var nn: NeuralNetwork

This is because the Neural Net depends on the fact that nn is not assigned anything.

When your AI or player has to be killed or removed, always use the queue_free() method. This is because the Neural Net relies on the signal emitted by the node when exiting the tree to recieve the fitness and Neural Network of that node. Example:

Object.queue_free()

What each variable means and how to use them

Ai Scene: This is where you will assign the AI or Player scene by clicking on the drop down arrow on the right side, clicking quick load and selecting your scene.
Batch Size: This is the informal Batch Size of each generation. The actual batch size of each generation is emitted by the true_batch_size signal. This controls the base amount of AIs spawned.
Generation Delay: This is the time limit (in seconds) for any generation. Once a generation has lived longer than the amount specified in this, the generation is reset and the next generation comes.
Input Nodes: This is where the input nodes for the nn will be set. Input Nodes means how many different inputs will the nn recieve.
Hidden Nodes: This is where the hidden nodes for the nn will be set. Hidden Nodes means how many nodes will process the data given by the input nodes. You should experiment with this amount.
Output Nodes: This is where you will set how many outputs you want to recieve by the nn.
Random Population: This determines how many AIs with random nn will be spawned after the first generation (after the 0 generation). It is a good idea to set this to a value greater than 10 as it allows for more possibilites to be explored by the Neural Net.
Use Reproduction: This determines whether reproduction will also be used to create new AIs for the next generations. This enables for combination of different traits of different nns. However, you will most probably not need this as Random and Mutated Population will suffice.
Reproduced Population: If “Use Reproduction” is checked, this will determine how many AIs will be spawned with reproduced nns. Note: This value must always be greater than half of the value of Batch Size if you have checked “Use Reproduction” as true.

How to use Neural Net

Just ensure that all the variables/properties mentioned above are correctly set. The position of this node is where all the AIs will be spawned, meaning, the position of this node = position of AI when spawned.

How to use Neural Network

var nn: NeuralNetwork = NeuralNetwork.new(input_nodes, hidden_nodes, output_nodes)

Input Nodes: This is where the input nodes for the nn will be set. Input Nodes means how many different inputs will the nn recieve.
Hidden Nodes: This is where the hidden nodes for the nn will be set. Hidden Nodes means how many nodes will process the data given by the input nodes. You should experiment with this amount.
Output Nodes: This is where you will set how many outputs you want to recieve by the nn.
If the Neural Network depends mostly on inputs from raycasts, you can use the “get_prediction_from_raycasts(optional_val: Array = [])”. This function returns an array of floats which are the outputs. The “optional_val” is optional can be used to give more custom inputs in addition to the raycasts. Example:

var output = nn.get_prediction_from_raycasts()

# or

var output = nn.get_prediction_from_raycasts([0, 0.4, 2])

You can use the predict(input_array: Array[float]) function also to get predictions. Example:

var output = nn.predict([0.0, 6, 0.2])

If you know the expected output of an input, you can use the train(input_array: Array, target_array: Array) function in a loop. Example:

for epoch in range(2000):

nn.train([0, 1], [1])

nn.train([1, 1], [1])

nn.train([0, 0], [0])

nn.train([1, 1], [0])

If you want to mutate your Neural Network, you can do so by:

nn = NeuralNetwork.mutate(nn)

where nn is your Neural Network.

If you want to mutate your Neural Network, you can do so by:

new_nn = NeuralNetwork.copy(nn)

where nn is your Neural Network and new_nn is the new one to which you are copying the nn to.

IF you want to reproduce your Neural Network with another, you can do so by:

reproduced_nn = NeuralNetwork.reproduce(nn_1, nn_2)

where nn_1 and nn_2 are the parent Neural Networks.

Q-Learning Algorithm

This algorithm implements Q-Learning algorithm using Q-Table natively in Godot.

How to use QLearning class

Initialise a QLearning variable
```
var qnet: QLearning = QLearning.new(observation_space, action_space)
```
Both the observation_space and action_space have to be natural numbers representing the possible states the agent can be in and the possible actions choices the agent can take in any given state.
Get a prediction from the QLearning variable:
```
qnet.predict(current_state, reward_of_previous_state)
```
The above method returns an whole number that lies between 0 and action_space - 1. The value returned corresponds to an action the agent can take.

You can assign the returned value to variable by:
```
var action_to_do: int = qnet.predict(current_state, previous_reward)
```

Configurable Values

qnet.exploration_probability -> has to be a float value

Default Value: 1.0

The probability that the agent will take a random action or exploit the data it has learned.

Do not change unless you know what you are doing.
qnet.exploration_decreasing_decay -> has to be a float value

Default Value: 0.01

Changes how the value by which the qnet.exploration_probability decreases every ```qnet.decay_per_steps`` steps.
qnet.min_exploration_probability -> has to be a float value

Default Value: 0.01

The minimum value the exploration_probability can take.
qnet.learning_rate -> has to be a float

Default Value:0.2

The rate at which the agent learns.
qnet.decay_per_steps -> has to be natural number

Default Value: 100

After how many steps does the qnet.exploration_probability decrease by qnet.exploration_decreasing_decay value.
qnet.is_learning -> has to be a bool value

Default Value: true

To be set to false only when the qnet.QTable.data is set manually.
print_debug_info -> has to be a bool value

Default Value: false

This can be set to true if you want to print debug info - total steps completed and current exploration probability - every qnet.decay_per_steps.

Things to keep in mind when using QLearning

The predict method of the QLearning class takes two compulsory arguments:
```
qnet.predict(current_state, previous_state_reward)
```
The current_state has to be a whole number representing the state it is currently in, while the previous_state_reward has to a float representing the reward it got for the previous action it took.

Minimax Algorithm

The implementation of this algorithm currently does not support alpha-beta pruning (however, depth-limited minimax is implemented but not documented), i.e., this algorithm is not optimised whatsoever. If the AI is playing the role of the adversary, then set minimax.is_adversary to true else false.

How to use Minimax class

Initialise the Minimax class with 4 arguments:
var minimax: Minimax = Minimax.new(Callable(result), Callable(terminal), Callable(utility), Callable(possible_actions))
1. result_func: Callable: This callable argument must link to the function in your code that returns the state of the environment after a particular action is performed.
2. terminal_func: Callable: This callable argument must link to the function in your code that returns true if the game is over and false if the game can continue for a given state.
3. utility_func: Callable: This callable argument must link to the function in your code that returns the value of the given state. Currently this function only runs when the game is a terminal state. Losing states should have lesser value than winning states.
4. possible_actions_func: Callable: This callable argument must link to the function in your code that returns all the possible actions for a given state.
5. Every time the AI needs to perform an action, call the action(state) on the minimax variable.
  var action_to_do: Array = minimax.action(_board)
Structure of the 4 arguments specified above
These functions have to be implemented by the user themselves as it is dependent on the game.
1. func result(state: Array, action: Array, is_adversary: bool) -> Array:
  Should return the resultant state from performing the action.
2. func terminal(state: Array) -> bool:
  Should return true if the no further action can take place, otherwise, it should return false.
3. func utility(state: Array, is_adversary: bool) -> float:
  Should return the value of the given state. Usually positive for states in which the AI wins and negative for states in which the AI lose.
4. func possible_actions(state: Array) -> Array[Array]:
  Should return all the possible actions that can happen in the given state. Each action is an array item inside the array that is being returned.
Look into the tictactoe demo to gain a better understanding.

ryash072007/Godot-AI-Kit