Drowsy is a school project aiming to build an AI architecture able to autonomously create a video game compatible with the Bitsy game engine.
It features :
- a GAN to generate Avatars
- a whole new Discriminator Network / Adversarial Genetic Algorithm coupling to generate Rooms (learn more)
The AI generated game is available to play here.
Here's the full project report (French). Below is an English summary.
For any questions, contact me at yrabii@ensc.fr, or on Twitter. I'm currently looking for opportunities to study AI and Computational Creativity, like an internship or a PhD. Please get in touch !
Made in 3 months during 2nd year of engineering school, at the École Nationale Supérieure de Cognitique (Bordeaux, 2019)
Bitsy avatar, representing a human shape
Ignoring color palette and animations, a Bitsy avatar is a 8x8 image with black and white pixels. The dataset is composed of 420 avatars, extracted from this compilation tweet.
Examples from the avatars dataset
Simplified structure of a GAN
The Adversarial part is composed of the Discriminator network and a Generator network layered together.
Since we process images, the chosen networks are CNNs. You can find more information about GANs here.
Outputs of the avatar GAN after 2 hours of training
-
Data shape
The Generator cannot produce binary data, but only real numbers in the range [0.0 - 1.0]. Produced avatars need a post-processing to convert non-binary gray pixels to a black and white palette.
-
Aesthetic
The shapes do look like Bitsy avatars. They are evocative, allowing the player to give them a meaning.
-
Diversity
Slight Mode Collapse around 2 or 3 classes at each training. Retraining just a few times switch the classes.
-
Performance
These performances where obtained after 2 hours of training on a laptop, with a GTX 1050 and 2 Go of RAM.
typical Bitsy room, representing a forest (Source)
Ignoring color palette, a Bitsy room is a 128x128 image with black and white pixels. The dataset is composed of 591 rooms, extracted with a custom scrapper used on this list of 21 Bitsy games.
The initial approach was to use the same GAN structure than the Avatar generation, only scaled to process 128x128 images.
Outputs of the Room GAN after 4 hours of training
-
Data shape
Similarly, produced images need to be post-processed to only keep binary pixels.
-
Diversity
Huge Mode Collapse.
-
Performance
These performances where obtained after 4 hours of training on a Google Colaboratory dedicated machine, with GPU acceleration.
-
Aesthetic
The shapes are structureless gradients, where Bitsy maps show these characteristics :
- local symmetry
- pattern repetition
- complex structures made of unitary pieces
The unsatisfactory previous results led us to rethink the architecture behind Room generation. Let's take another angle. We're looking for these characteristics :
- local symmetry
- pattern repetition
- complex structures made of unitary pieces
They are commonly found in the realms of Cellular Automata.
Example of cave-like level generation using cellular automata (Source)
Cellular Automata are frequently used to procedurally generate video game levels. They iteratively apply local transformation rules to an initial grid until it has specific characteristics. Instead of directly handling all the pixels to create a room, we could manipulate the rules of a Cellular Automaton that generates room-like images.
The main issue is that the algorithm running a CA is exact, well-known and simple whereas our current architecture is based on neural networks running complex and evolving algorithms, relevant for unpredictable cases.
There's a huge mismatch between the features we expect of the Adversarial half, and it's nature. We need a structure both able to :
- generate images following a specific algorithm (Cellular Automata)
- iteratively approach a specific criterion (fooling the Discriminator)
Our approach is to change the Adversarial half from being a neural network to a genetic algorithm. The requirements to use such a structure are :
- being able to represent data as a set of genes
- having a way to evaluate their quality with a fitness function
In our case, the genes are the rules used to generate a room. They consist of the rules of a CA and the rules to generate the initial population of the grid (eg : the two parameters of a normal distribution). The key point is using the evaluation function of the Discriminator as the fitness function of our new Adversarial component.
Structure of the DN/AGA coupling
If we see GANs as a coupling between a Discriminator Network and an Antagonist Network (DN/AN), we can describe our new architecture as an asymmetric coupling between a Discriminator Network and an Adversarial Genetic Algorithm (DN/AGA).
Rooms generated by CA rules selected by the DN/AGA coupling
-
Data shape
The generated rooms can directly be used a Bitsy rooms, without post-processing.
-
Diversity
The selected room rules by the AGA are diverse, there's no observed phenomenon equivalent to Mode Collapse. One selected set of rules can be reused multiple times to produce rooms with similar features, while being a different room. Simply changing the rules behind the generation of the CA initial population can heavily impact the style of the room . Eg: from Normal Distribution (down right room) to Uniform Distribution (top left room).
-
Performance
These performances where obtained after 2 hours of training on a PC, with a GTX 1050 and 2 Go of RAM. (Same machine as in the Avatar generation)
-
Aesthetic
We observe the desired characteristics :
- local symmetry
- pattern repetition
- complex structures made of unitary pieces
Furthermore, the generated rooms reproduce structures often found in Bitsy games.
- top right : maze-like room, made of several corridors
- top left : museum-like room, with distinct items scattered in the space
- bottom left : road leading to another room
- bottom right : huge central structure
More examples can be found in the archives.
The final game was assembled by compiling together a selection of rooms generated by several cellular automata, whose rules were designed after 1 hour of training by the DN/AGA coupling. The player's avatar was generated by a GAN, after 2 hours of training. We then manually added doors to go from one room to another, as well as an introduction and conclusion text.
The DN/AGA coupling was experimentally developed to go beyond the constraint of having a Generator in the form of a neural network. This structure generalizes the theory behind classical GANs and extends its scope of application to fields where the use of explicit algorithms is relevant. To our knowledge, this kind of coupling between a Discriminator Network and an Adversarial Genetic Algorithm is yet to be studied academically.
Here are some observations we made empirically :
-
Costs
The DN/AGA structure appears to be way less costly in computation time than its GAN counterpart, and can be done with cheaper hardware
-
Training
Classical GAN training is made by alternating turns : training the DN on a batch, then the AN, and repeat. Better results were obtained for the DN/AGA by letting each part train enough times before handing over to the other. Criteria for determining if more training was needed were :
- for the Discriminator Network : an accuracy below a specific threshold
- for the Adversarial Genetic Algorithm : the median fitness of the population below a specific threshold
For any questions, contact me at yrabii@ensc.fr, or on Twitter. I'm currently looking for opportunities to study AI and Computational Creativity, like an internship or a PhD. Please get in touch !