Implementation of the Lamba_u is not correct

[wang3702]

_match *= tf.clip_by_value(tf.cast(self.step, tf.float32) / (warmup_kimg << 10), 0, 1)
here warmup_kimg=1024, that's to say, it should be _match%=step/1048576
Yours:
def linear_rampup(current, rampup_length=16):
if rampup_length == 0:
return 1.0
else:
current = np.clip(current / rampup_length, 0.0, 1.0)
return float(current)

[YU1ut]

1048576/(64-batch_size * 1024-iteration_per_epoch) = 16-epoch. Am I wrong?
The step in their code is updated by

self.ops.update_step = tf.assign_add(self.step, FLAGS.batch)

https://github.com/google-research/mixmatch/blob/master/libml/train.py#L51

[wang3702]

For your batch size, no. However, considering the situation user will give different batch_size. Therefore, your updating strategy is not correct.
Also, please notice google's actual training batch size is also different from you.(See their paper)

[YU1ut]

OK. I will fix it. But training batch size is always 64 in all experiments.
https://github.com/google-research/mixmatch/blob/master/mixmatch.py#L144