Singular matrix during inversion

[xuesongwang]

Hi there,
Thank you for this amazing project. I was using this ADHD200 dataset to remove site effects. I computed Pearson coefficients for AAL functional connectivities as inputs and some sites have provided biological features such as gender, handedness and IQs. However, when calling the neuroCombat , I got the following error on this function get_beta_with_nan:
LinAlgError:Singular matrix

neuroCombat/neuroCombat/neuroCombat.py

Lines 211 to 221 in ac82a06

	def get_beta_with_nan(yy, mod):
	wh = np.isfinite(yy)
	mod = mod[wh,:]
	yy = yy[wh]
	B = np.dot(np.dot(la.inv(np.dot(mod.T, mod)), mod.T), yy.T)
	return B
	betas = []
	for i in range(X.shape[0]):
	betas.append(get_beta_with_nan(X[i,:], design))
	B_hat = np.vstack(betas).T

I am no experts in maths, but according to this solution, it can be attributed to the same rows in the matrix so the inversion does not exist. I didn't fully understand this part of the code, but I guess I can have some patients who have the same biological features and that caused the issue? (assumption though). After I changed la.inv into la.pinv everything works fine now. I was wondering if you could have better insights towards this and whether modification is solid.

Thanks for the help :)

[Jfortin1]

Hi @xuesongwang,

LinAlgError:Singular matrix usually means that some of the biological features are redundant (perfectly confounded) or highly correlated. Is this the case?

[Jfortin1]

Closing this as no answer from user.

[xuesongwang]

Sorry for the late but you are are right @Jfortin1 , I have one column called IQ measure and that can probably be inferred from three other columns. That might have result in zero determinant matrix or redundant columns (hence non-invertible). many thanks