cjlin1/libmf

Possible overflow in matrix access

AnnSeidel opened this issue · 0 comments

When trying to factorize a large matrix, I had a lot of weird errors in my calculations, which turned out to be due to seemingly random access to the model matrices P and Q. I found that in the calc_reg1_core function, there is a missing cast to mf_long in the matrix access:

for(mf_int j = 0; j < model.k; j++)
    tmp += abs(ptr[i*model.k+j]);

needs to be

for(mf_int j = 0; j < model.k; j++)
    tmp += abs(ptr[(mf_long)i*model.k+j]);

since i*model.k+j could be larger than INT_MAX