Sparse Input Data

Question

Sparse Input Data

3bst0r opened this issue 8 years ago · 2 comments

Is there a way to input sparse data? I suspect this is not a straight-forward thing to do, because of the lack of a standard way to store sparse matrices in a text file, i.e. python probably does it different than matlab (did not check though).

OT: I just watched a video of you presenting t-SNE at Google and I want to compliment you on your explanation skills. Very clear and understandable.

Answer 1 · 2016-09-13T13:33:34.000Z

This is not implemented right now. Both Matlab and Numpy support compressed sparse row matrices, so it would be possible to add this.

For now, a potential way to circumvent this would be to do some kind of (logistic) PCA preprocessing in Matlab / Python and use the reduced data as input into t-SNE. (This is assuming that the full matrix is too big to keep in memory.)

Answer 2 · 2017-03-09T06:31:29.000Z

If I´m not mistaken, Truncated SVD is also good for preprocessing sparse data and it can work with scipy.sparse.

http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html

(I am brand new to the field, this is my first ever comment on github - breaking the ice here ;)).