Different output activation function

[rickstaa]

When trying to solve rickstaa/LAC-TF2-TORCH-translation#38 I found out that the old TensorFlow version uses a relu activation function as the output activation function for the fully connected layers of the Gaussian Actor and Lyapunov Critic. The following output activation functions are used in literature:

Gaussian Actor output tfunction

• Minghoas code uses the Relu function.
• Spinning also uses the Relu function.
• Haarnoja et al. 2019 use multiple functions. For the normal gaussian actor they use the identity function. For other versions, they use the nn.tanh function.

Lyapunov critic

• Minghoas code uses the square function as the output activation and the relu in the last lidden layer.
• Spinning up uses the Identity function.
• Haarnoja uses the identity function.

When inspecting the structures further Minghoas version is equal to Haarnojas version the output layer now however becomes a hidden layer and the Square activation function is used after that. The package already contains a method to change the activation functions from the CLI.

[rickstaa]

All the different output activation functions allow the agent to train. Based on the task and environment they might however still infleunce the performance.