This project builds upon the existing neat-python library created by CodeReclaimers. Details about the original NEAT algorithm implementation can be found on CodeReclaimer's neat-python description. The purpose of this fork is to add the Novelty Search algorithm to the neat-python library.
The Novelty Search algorithm, developed by Joel Lehman and Kenneth O. Stanley, is an unsupervised learning algorithm that is driven by exploration of novel behavior. Instead of measuring an individuals performance with an objective function, individuals are rewarded for their measured novelty. This is similar to the idea of curiosity in reinforcement learning and robotics.
The specific implementation of the Novelty Search algorithm is detailed in "implementation_description.pdf".
For easy adaptation, enabling the novelty search algorithm is similar to the general use of the neat-python library. The user must provide novelty search parameters (summarised below) in the configuration file that lists all NEAT experiment settings. The user must also provide an evaluation function that calculates each individual’s behavior, similar to the function that calculates an individual’s fitness.
The configuration parameters are summarised below:
novelty_search_enabled (boolean): If set to True, the novelty search algorithm is enabled.
pop_knn_neighbours (integer): The number of nearest neighbours to consider when calculating the distance of an individual’s behavior to other behaviors in the same population.
archive_knn_neighbours (integer): The number of nearest neighbours to consider when calculating the distance of a individual’s behavior to other behaviors in the novelty archive. Typically lower than pop_knn_neighbours as novelty archive has fewer members than a population.
threshold (float): If the knn distance of a generation’s most novel individual in the novelty archive exceeds this threshold, it is added to the archive.
As it is critical to not break functionality of existing NEAT experiments, the algorithm parameters default to none when no novelty search parameters are provided in the configuration file.
- An example case of the novelty search algorithm.