decay for sigma can be improved

[BrandonGarciaWx]

Hey,

I have a few small tweaks to the MiniSom class that should help users stick to the best SOM practices.

1. I suggest removing the following code from the "__init__" constructor method:
   if sigma >= x or sigma >= y: warn('Warning: sigma is too high for the dimension of the map.')
   Oftentimes, it is useful to have sigma equal to the hypotenuse of the SOM during the first round of training so that all neighboring nodes are significantly updated towards the beginning of the training. Using a sigma value equal to the hypotenuse of a 3x4 SOM (which is 5) would currently yield this warning despite it being a safe practice.

2. For the "__init__" constructor method, I suggest breaking up the "decay_function" argument into separate decay functions for the learning rate and radius of influence. My reasoning for this is that typical SOM practices often call for the learning rate to decrease to a value of zero by the end of the training session. However, decaying sigma to zero often results in overfitting.
   I suggest using an inverse function like learning_rate(t) = learning_rate / (1 + 100(t/num_iteration)) for the default learning rate decay function (similar to the inverse-time function under the "alpha_type" parameter on page 14 of the attached SOM_PAK document SOMPAK_1996.pdf).
   A common practice for avoiding this overfitting is to decrease sigma to a value of one by the end of the training session instead of zero (similar to the description under the "radius" parameter on page 13 of the attached SOM_PAK document SOMPAK_1996.pdf). I suggest using an inverse function like sigma(t) = sigma / (1+t/(num_iteration/(hypotenuse - 1)) for the default sigma decay function.
   The current default decay function struggles to converge on solutions for sigma values greater than 3 due to the sigma at the end of the training being larger than 1 which results in less fine-scale tuning that usually occurs when the sigma value is near a value of 1.

Thank you for your consideration!

[JustGlowing]

These are great suggestions, would you like to raise a pull request implementing them?

[JustGlowing]

Never mind, I just pushed an update that allows you to pick a different decay function for sigma and implemented the functions mentioned in the sompak documentation you linked.

[vwgeiser]

@JustGlowing I downloaded Minisom through pypi this week and it appears the sigma_decay_function isn't in the MiniSom constructor? I checked the version and it says 2.3.2 so I don't appear to have an outdated version.

Even though it is in the minisom.py master does this need to be updated for the pypi distribution?

[JustGlowing]

Hi @vwgeiser those changes have not been released to pipy yet, you can install the development version of minisom as follows

git clone https://github.com/JustGlowing/minisom.git
python setup.py install

[vwgeiser]

@JustGlowing Gotcha, sounds good.