This repository hosts the neural network component of Rose-Hulman’s AI XPrize robot. For more detailed information of the project as a whole, see this repo.
To get started, first pip3 install --user -r requirements.txt. Run
generate_text_instructions.py to create sample input and output
files for the model to train on.
Next, run each of text_flip_trainer.py, text_colors_trainer.py,
and text_letters_trainer.py on the appropriate output files to train
the model.
Finally, either run model_reply.py to hop into a REPL and test the
model, or use the model_runner.py interface in another program to
use the generated models.
generate_text_instructions.py <num-to-generate> <neural-infile> <neural-flips-outfile> <neural-colors-outfile> <neural-letters-outfile>
Generates <num-to-generate> sample instructions and meanings to the
four .csv files <neural-infile>, <neural-flips-outfile>,
<neural-colors-outfile>, and
<neural-letters-outfile>.
<neural-infile>holds the raw text (in all caps to reduce the search space)<neural-flips-outfile>holds a boolean matrix of instruction types:
| Flip | Move |
|---|
<neural-colors-outfile>holds a boolean matrix of colors:
| Red | Green | Blue | Orange | Yellow | None |
|---|
<neural-letters-outfile>holds a similar boolean matrix of letters:
| A | B | C | D | E | F | G | H | None |
|---|
These each signify the desired meaning of the corresponding line of text. The ‘None’ columns mean that no color/letter was detected (e.g. “Move the blue block” has no letter).
text_flip_trainer.py <text-file> <flips-file> <model-output> (<model-input>)
Trains a neural network and stores it in the HDF5 file
<model-output>. The output is written out once every ten epochs. If
<model-input> is provided, then it will load the weights stored in
that HDF5 file before continuing training.
<text-file> is a csv file with a list of text inputs from Generate
Text Instructions.
<flips-file> is a csv file with a boolean matrix of expected flips
outputs from Generate Text Instructions.
text_colors_trainer.py <text-file> <colors-file> <model-output> (<model-input>)
Trains a neural network and stores it in the HDF5 file
<model-output>. The output is written out once every ten epochs. If
<model-input> is provided, then it will load the weights stored in
that HDF5 file before continuing training.
<text-file> is a csv file with a list of text inputs from Generate
Text Instructions.
<colors-file> is a csv file with a boolean matrix of expected colors
outputs from Generate Text Instructions.
text_letters_trainer.py <text-file> <letters-file> <model-output> (<model-input>)
Trains a neural network and stores it in the HDF5 file
<model-output>. The output is written out once every ten epochs. If
<model-input> is provided, then it will load the weights stored in
that HDF5 file before continuing training.
<text-file> is a csv file with a list of text inputs from Generate
Text Instructions.
<letters-file> is a csv file with a boolean matrix of expected
letters outputs from Generate Text Instructions.
model_repl.py <flips-model.h5> <colors-model.h5> <letters-model.h5>
Loads up the model files in <flips-model.h5>, <colors-model.h5>,
and <letters-model.h5> and spins up a simple REPL instance to test
the model. Sample usage looks like this:
> Move the red a here
('Move', 0.99999213)
('red', 0.9981108)
('A', 0.9987594)
> Flip the blue block
('Flip', 0.9937981)
('blue', 0.99789387)
('None', 0.9977901)
>
It outputs three tuples, each of which holds the guessed meaning and the output (out of one) at that output neuron. The first is whether to move or to flip, the second is the color, and the third is the letter.