[W reducer.cpp:320] Warning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance.
Closed this issue · 1 comments
DY-ATL commented
Hello, do you have any idea why this happen?
[W reducer.cpp:320] Warning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance.
grad.sizes() = [256, 256, 1, 1], strides() = [256, 1, 256, 256]
bucket_view.sizes() = [256, 256, 1, 1], strides() = [256, 1, 1, 1] (function operator())
PhilippeWeinzaepfel commented
Hi,
I have never seen such message. Can you let us know the command you have launched as well as maybe the pytorch/cuda versions ?
Best
Philippe