In this repo, we will be exporing Hopfield networkds as an alternative to the traditional approach to attention in transformer models.
Make two calsses of "molecules" with a couple gaussian spectral features.
Use those features to select 5 random electron counts of the 256 nonuniform quantized bins.
Try to use the Wasserstein distance to determine "similar" and "dis-similar" binary vectors.
Split the similar vectors across the Hopfield heads and train each head from only a few of each dis-similar training sets.
Then feed combinations of multiple molecule classes with ~25-50 counts in order to see if the multi-head hopfield can get the relative ratios right.
Give it a new molecule and see if it can recover the input, run it like an auto-encoder.
Later, try doing nonuniform quantization.