sync-bn efficiency?
hszhao opened this issue · 0 comments
hszhao commented
Hi, thanks for your work on sync-bn.
1.I want to know the efficiency comparison between un-sync bn and your implementation of sync-bn?
2. Btw, has anyone meet program stuck (I meet this at some iterations when 2 gpus are used and at the beginning if 4 gpus are adopted)?