Determine optimal mini batch size

[tallamjr]

If the size of the training set is not evenly divisible by the a batch size number, then the final batch that is trained will be trained on less items and affect the performance.

There should exist a formula to optimally choose a batch size that allows for a maximal number of samples to exist in the final batch size.

[tallamjr]

train_len = 3990

batch_size_list = [16, 32, 64]
ratios = []
for batch_size in batch_size_list:

    remainder = train_len % batch_size

    if remainder == 0:
        batch_size = remainder
    else:
        ratios.append(batch_size / remainder)

index, ratio = min(enumerate(ratios), key=lambda x: abs(x[1] - 1))

print(batch_size_list[index])

Ref: https://stackoverflow.com/questions/9706041/finding-index-of-an-item-closest-to-the-value-in-a-list-thats-not-entirely-sort

>>> 1024/918
1.1154684095860568
>>> 512/406
1.2610837438423645
(astronet) 20:47:04 ✔ ~/scratch  :: python batch.py
(4, 1.1154684095860568)
(astronet) 20:35:44 ✔ ~/github/tallamjr/origin/astronet (issues/53/load-datasets) :: python
Python 3.7.8 | packaged by conda-forge | (default, Jul 31 2020, 02:37:09)
[Clang 10.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 3990 % 2048
1942
>>> 2048 / 1942
1.054582904222451
>>> 3090 % 2048
1042
>>> 2048 / 1043
1.9635666347075742
>>> quit()

[tallamjr]

Closed with e8ecc89