See if multi-start solves the AUC vs. component # jitter
Closed this issue · 5 comments
Setup a method that runs the PARAFAC2 fitting a few times and takes the result with highest R2X.
@aarmey - Tested this a bit. It appears that the R2X/error has no consistent effect on the AUC - so the multi start doesn't get rid of the jitter.
Ok. How did you test it?
Two different ways. First I remade the graph of AUC vs components but for each component number did 10 decompositions and took the one with the smallest reconstruction error and used it. Then I tried doing 10 decompositions with 30 components (and a few others) and Showing the errors and auc values using both. The magnitude of the error did not show any obvious trend vs the auc.
Ok, the other thing I'd try is lowering the decomposition tolerance, perhaps to tol=1e-9. It'd be helpful to know, when you have ~38 components, how many iterations it's taking to fit, which you can get some information about using verbose=True. If you're hitting the iteration limit then you can pass n_iter_max=1000. This will take longer to fit, but when we move to running it on the GPU it probably won't slow down much.
https://github.com/tensorly/tensorly/blob/master/tensorly/decomposition/parafac2.py#L130
With tol=1e-8 it takes about 400-500 iterations to converge.