Possible overflow in matrix access
AnnSeidel opened this issue · 0 comments
AnnSeidel commented
When trying to factorize a large matrix, I had a lot of weird errors in my calculations, which turned out to be due to seemingly random access to the model matrices P and Q. I found that in the calc_reg1_core
function, there is a missing cast to mf_long
in the matrix access:
for(mf_int j = 0; j < model.k; j++)
tmp += abs(ptr[i*model.k+j]);
needs to be
for(mf_int j = 0; j < model.k; j++)
tmp += abs(ptr[(mf_long)i*model.k+j]);
since i*model.k+j
could be larger than INT_MAX