denizyuret/Knet.jl

Error calculating cat gradient on GPU

fabricerosay opened this issue · 4 comments

let

t(x)=sum(cat(x,x,dims=3))

Then julia grad(t)(rand(6,6,1,1)) works.
But julia grad(t)(KnetArray(rand(6,6,1,1))) returns the following errors CuArray has no field ptr.

Following the error message leads to this funtion in karray.jl

function KnetArray(x::CuArray{T,N}) where {T,N}
    p = Base.bitcast(Cptr, x.ptr)
    k = KnetPtr(p, sizeof(x), Int(CUDA.device().handle), x)
    KnetArray{T,N}(k, size(x))
end

If I change it with:

function KnetArray(x::CuArray{T,N}) where {T,N}
    p = Base.bitcast(Cptr, x.baseptr)
    k = KnetPtr(p, sizeof(x), Int(CUDA.device().handle), x)
    KnetArray{T,N}(k, size(x))
end

Then the calculation works, the field ptr in Cuarrays has changed to baseptr with CUDA.jl
Would this work as intended ?

Fixed indeed the CUDNN errors in LSTM (as long as using a newer version of CUDA). Thanks!!!

Similar issue when upgrading to CUDA 2.2.1 -- not just LSTM, but also in my 3D DenseNet.

It seems like CuArray field name changed from ptr to baseptr in CUDA@2.1.0.
Using pointer(a) seems to work in both cases, I will fix this and check for other cases of a.ptr in the code.

#636 fixes this.