google-deepmind/deepmind-research

stochdepth_rate in NFNets

YilanWang opened this issue · 1 comments

YilanWang commented

thanks for this great job, and I have a question, the block_stochdepth_rate in nfnets is:
block_stochdepth_rate = stochdepth_rate * index / num_blocks
but the index is set to 0 in

deepmind-research/nfnets/nf_resnet.py

Line 64 in f5de0ed

index = 0 # Overall block index

So the block_stochdepth_rate is always 0, isn't it?
thanks!