This repository served as a container for notebooks investigating the development of novel quantum convolutional neural networks for foreccasting biogas production during anaerobic processes in waste-management facilites.
No software installations are necessary for executing the models. Simply import the notebooks into Google Colab to execute code. The primary notebooks, ModelMaker, QuantumModelMaker, and Processor follow two distinct but congruent processes. Using workflow design patterns preprocessing and, validation and model building are separated and containerised. Checkpoint pattern is used to reduce the chances of overfitting and provide some tolerance to delicate architectures like QCNNs.
The PreProcessor notebooks generates a dataset that can be utilised to obtain a continuous time series dataset. The final dataset, named viable_dataset.csv, is used in both model makers to obtain insights.
Both, discrete and time-window models for classical and quantum architectures can be made by using 'time-window' or 'discrete' as input arguments passed into buildmodel() functions.
Insights are obtained by executing validation functions within ModelMaker notebooks and are stored in the filesystem within Google Colab.
The software for this project developed entirely in Python 3.8. The development was done in an Ubuntu 18.04.5 LTS subsystem on Google Colab. Due to the conflicting mathematical operations performed by Tensorflow and Tensorflow Quantum, two sanitised classical and quantum development environment were utilised. Tensorflow and Keras were the primary libraries used for the development of classical models. Tensorflow Quantum and Cirq were used for developing quantum machine learning models. Pandas, Numpy, and Scikit-Learn were the primary libraries used for data preprocessing. Seaborn and Matplotlib were used for generating data visualisations.
The entire code base was developed on Google Colab. The execution and training of models does not require the installation of any packages and can be done entirely in Google Colab. Data files used can be found within \data, and the documentation for the code base can be found in \docs. All models developed during this investigatory project can be found within the \models subfolder while the performance metrics for all developed models can be found in the \results subfolder.
The entire codebase is contained within .ipynb notebooks detailing the data investigation, model development, and evaluation of models.
A combination of workflow pipeline and checkpoint design pattern was utilised for model development. Due to the delicate structure of QCNNs, resilience and fault tolerance were prioritised. Workflow pipeline design pattern was used to ensure that the models were reproducible. In this pattern, each step in the machine learning workflow, preprocessing, model training, and validation, was isolated and containerized into three distinct sections. The checkpoint was used because of this pattern's unique ability to reduce overfitting by saving an instances of the model with the lowest validation loss during training. The preprocessing layer removed features with high sparsity and low correlation. Data imputation by replacing missing values with Random Forest regressors was applied to features with data density above the accepted tolerance of 60%. For models that utilised a short-term memory for forecasting, training data was made stationary with exponential smoothing methods by using the StatsModel API. Finally, for models built on a quantum architecture, dense angle encoding, was applied with Cirq.
Both classical and quantum models were built using Tensorflow and Tensorflow Quantum in Python within the model training section. The checkpoint design pattern was utilised to prioritise fault tolerance and resilience for all models. After each epoch of training, the validation loss for the model was measured. If the validation loss was lower than that of the previous epoch, a snapshot of the model's state was saved to be able to resume training in the future.A so-called patience time, the time before the ML-training process is aborted due to no improvement in validation loss, was set to 10 epochs. This allows sufficient time for the algorithm to potentially escape local minima in the optimization process The data was continuously validated to maintain constant dimensional sizes for the dataset and separation among the training and testing data.
Finally, the validation Section initialised the models saved after model training to obtain key metrics detailed in the Methodology section. These metrics, along with the forecasts generated, were saved within the \results subfolder.