LSTM architecture to evaluate sequences of images based off the Oxford LSTM Tutorial practical 1 repository. For each property, we have multiple images and one label, corresponding to Selling Price decile or some other related task. This architecture analyzes each image and then outputs the class label at the end as depicted in the schematic:
![Schematic] (LSTM_model.png)
There are two basic Lua functions, one for training train_lstm.lua
and one for testing the performance of the model test_lstm.lua
.
In the raw/
directory, store the train and test CSVs separately. For training, the CSVs should be of the format
PropertyID | ImageID | Class Label | Representation Columns (input) |
---|---|---|---|
propertyid_1 | imageid_1 | class_int | Numbers |
propertyid_2 | imageid_2 | class_int | Numbers |
With this format, we may excecute the R-script ReplicateImages
, which creates fixed length sequences of images for the purpose of training. Specifically, it groups all properties together and repeats the images associated with the property until it reaches the fixed sequence length. These CSVs may be now stored in the data/
directory and can now be converted to the necessary format.
Both train_lstm.lua
and test_lstm.lua
operate on multidimensional Torch Tensors, so we must convert our CSVs into this format with the CreateTensors.lua
script. The CreateTensors.lua
script has a function, generateTensors which takes as argument the base-name for the CSV file and the relative reference paths to the directory for the CSV files and the outputted tensors. It is only necesary to run this when the input CSVs have not yet been converted to Torch Tensors.
From main directory, one can now call from the command line
th train_lstm.lua
which will execute over the inputted files specified in the opt
Lua table. Additional specifications are provided here, however, one must match the following criteria in the table:
- Sequence length
opt.seq_length
must match the sequence length declared in R - The input size must
opt.input_size
must match the dimension of the Representation Chosen - The output size
opt.output_size
must match the number of classes provided (classes must be 1-indexed!!)
The LSTM architecture may be tested by loading a trained model from the snapshots/
directory and then executing
th test_lstm.lua
Properties of testing are set in the opt
Lua table, similar to the one found in train_lstm.lua
.
- Representation is sparse at the hidden layer FC7, concerning?
- Fix the max-sequence for each image (training and testing) and just generally devise a better strategy here.
- Automatically detect if the torch tensors need to be created at the beginning of train/test