Associative Learning with Sparsity Inducing Constraints
These functions generate the replica theoretical and numerical solutions for the constrained perceptron learning models described in the manuscript entitled “Sparse learning enabled by constraints on connectivity and function” by Junaid Baig and Armen Stepanyants. They incorporate $ℓ_0$, $ℓ_1$, $sign$, and $gap$ constraints on connection weights, fixed firing threshold, and robustness to noise. The numerical methods based on integer and mixed-integer linear optimization can be used to find exact solutions for relatively small models, $N$ ~ 100. The sparse learning rule can generate approximate solutions online and works for larger $N$. The replica theoretic results, valid in the $N$ → ∞ limit, serve as benchmarks for all models and parameter settings.