MTNN (Multilevel Neural Networks) is a PyTorch-based library for the application of multilevel algorithms to the training of neural networks.
The algorithms behind MTNN mathematically work whenever the neural network can be decomposed into a set of operational subsets such that each subset consists of a set of neurons, channels, or other similar operational units. In this case, pairs of neurons (or channels or other operational units) can be matched up and restricted into a coarse network.
MTNN currently has software support for feedforward neural networks with
-Fully-connected layers
-One or more convolutional layers followed by 0 or more fully-connected layers.
Requires Python 3.6 or up
MTNN Dependencies:
- PyTorch
- Numpy
- Matplotlib (for plotting results)
The Anaconda Python distribution, free for individuals, has all of these.
To install MTNN, from the MTNN root, run the command
pip install -e .
Additional details about the algorithm and parameter settings can be found in ABOUT.md
Documentation can be generated via Doxygen with the commands
doxygen docs/Doxyfile
Datasets needed to run the Darcy and Poisson examples can be downloaded here: [link to dataset]
Place the data folders in /examples/datasets
There are a number of examples in the examples folder which
illustrate how to set up and train hierarchies of neural networks.
-circle_example.py is the best starting point, and shows visually
the regularization benefits of MTNN.
-darcy_example.py and poisson_example.py show using MTNN to
effective learn function estimation based on PDEs.
-mnist_example.py shows using MTNN for a classification task. MTNN
seems to work best here when the network of interest is squeezed
between a bigger and a smaller network in the multilevel hierarchy.
MTNN includes a visualization script that parses its log files and
plots results using matplotlib. This can be found in
MTNN/visualize_results.py.
By agreeing to contribute to MTNN, you agree to abide the rules in Code of Conduct
This work was produced at Lawrence Livermore National Laboratory:
-
Ruipeng Li (li50@llnl.gov)
-
Christina Mao (mao6@llnl.gov)
-
Colin Ponce (ponce11@llnl.gov)
This project is licensed under the MIT License. Please see LICENSE and NOTICE for details. All new contributions must be made under the MIT License.
SPDX-License-Identifier: MIT
LLNL-CODE-827581