backward pass
cnzhanj opened this issue · 7 comments
Hi @Lyken17
It looks like that the train data go through forward pass in checkpoint.py(function forward), But, when I use loss.backward, it looks like did not call backward function in checkpoint.py, instead it calls the initial backward function in torch.autograd.function.py? and the backward is not working because I print the model parameter values after each iteration. what's more, the speed in Subliner is faster than Sequential?
That is, have you reconstructthe backward function?
when I use loss.backward, it looks like did not call backward function in checkpoint.py
The backward function is modified. You can check the codes around https://github.com/Lyken17/pytorch-memonger/blob/master/momonger/checkpoint.py#L62 If it is not working, please attach a minimal code for reproducing.
and the backward is not working because I print the model parameter values after each iteration
Not sure what your case is.
the speed in Subliner is faster than Sequential
The sublinear should be slower than sequential.
Thank you!
I use the example resnet in momonger, and modify the code (add loss.backward) and it did call backward function, but as for other model it didn't, what's the error?
and I found, as for other model, it should change the output size? Is this cause the error?
@Lyken17
it is necessary to add the torch.flatten for align the tensors? Because the error occurs in alexnet:
output = input.matmul(weight.t())
RuntimeError: size mismatch, m1: [1536 x 6], m2: [9216 x 4096] at
Sublinear only supports nn.Sequential. Other operations are not considered at the moment.
Thank you! I will modify the vgg code and add nn.Sequential to the model as to suit for Sublinear , But it seems not work?
When something does work, show me the code (minimal case for reproducement) instead of simply telling me it. I am not a magician and cannot read your mind.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.