nn.SpatialSoftMax breaking

Question

nn.SpatialSoftMax breaking

shubhamjain0594 opened this issue 8 years ago · 9 comments

require 'nn'
require 'cunn'

softOutCalc = nn.Sequential():add(nn.SpatialSoftMax())
softOutCalc = softOutCalc:cuda()
ips = torch.rand(8,2,80,80)
ips = ips:cuda()
softOutCalc:forward(ips)

This code gives the error

THCudaCheck FAIL file=/tmp/user/1012/luarocks_cutorch-scm-1-199/cutorch/lib/THC/generic/THCTensorCopy.c line=81 error=77 : an illegal memory access was encountered
/home/users/shubhamj/torch/install/bin/luajit: cuda runtime error (77) : an illegal memory access was encountered at /tmp/user/1012/luarocks_cutorch-scm-1-199/cutorch/lib/THC/generic/THCStorage.c:158

It used to work fine on earlier installations.

Answer 1 · 2016-06-07T18:44:17.000Z

I just tried the snippet using most recent installations of cutorch and cunn and it works fine for me.

Answer 2 · 2016-06-09T10:29:23.000Z

I have updated to the latest version. Still getting same errors. Any help is appreciated.
I am working with cuda7.5 and latest released installs of torch, cutorch, nn, cunn and cudnn.

Meanwhile I also found that it works if I use

ips = torch.rand(1,2,80,80)

Answer 3 · 2016-07-28T02:27:42.000Z

Shubham, is this still a blocker? if so i can have a look.

Give me info on your GPU model (of all GPUs on system) and NVIDIA driver.

Answer 4 · 2016-07-28T07:38:34.000Z

Its working fine. Thanks

Answer 5 · 2016-08-02T07:44:10.000Z

I have the exact same issue.

System config:
Updated torch using '$ ./update.sh' and '$ ./install.sh'
Titan X, Ubuntu 14.04, cuda7.5, cudnnv5

@shubhamjain0594 : can you reopen the issue

Answer 6 · 2016-08-02T07:50:18.000Z

Let me give more cases with cudnn:

require 'nn'
require 'cunn'
require 'cudnn'

softOutCalc = nn.Sequential():add(nn.SpatialSoftMax())
softOutCalc = softOutCalc:cuda()
softOutCalc = cudnn.convert(softOutCalc, cudnn)  
ips = torch.rand(8,2,80,80)
ips = ips:cuda()
softOutCalc:forward(ips)

Gives

THCudaCheck FAIL file=/home/users/sasank/torch/extra/cutorch/lib/THC/generic/THCStorage.c line=147 error=77 : an illegal memory access was encountered

While,

require 'nn'
require 'cunn'
require 'cudnn'

softOutCalc = nn.Sequential():add(cudnn.SpatialSoftMax())
softOutCalc = softOutCalc:cuda()
ips = torch.rand(8,2,80,80)
ips = ips:cuda()
softOutCalc:forward(ips)

Works

Answer 7 · 2016-08-02T14:55:03.000Z

oh cool. so cudnn.convert is screwing something up. that's very helpful to figure out a fix. thanks.

Answer 8 · 2016-08-02T14:57:12.000Z

ok nevermind, it's not even cudnn.convert. the code has a bug.

Answer 9 · 2016-08-02T15:50:48.000Z

pushed a fix. reinstalling cunn fixes it.