DxD instead of NxN in the computation of the covariance matrix of Z?

Question

DxD instead of NxN in the computation of the covariance matrix of Z?

aarontyliu opened this issue a year ago · 2 comments

Hi, I came across the implementation of the covariance matrix of Z while trying to use VICReg in my project. I feel like I am missing something here, but I wonder if the implementation should be x @ x.T instead of x.T @ x?

Thanks for any information you can provide.

Answer 1 · 2023-10-23T13:10:42.000Z

Same quetions. Did you figure out? I also think the former implementation is right.

Answer 2 · 2023-11-01T12:08:55.000Z

The implementation should be x.T @ x, if x's dimension is (B, D), here B: batch_size, D: dimension, because they want the covariance matrix to be a function of the dimension