Fit didn't converge with warning: NEW_X
Closed this issue · 9 comments
Hello @stefanches7!
Thanks for the quick response - it helped me to run OUTRIDER.
With any input, including the input provided in the vignette, I receive the warning
> ods <- OUTRIDER(ods)
Fri Jul 27 23:30:45 2018: SizeFactor estimation ...
Fri Jul 27 23:30:45 2018: Running auto correct ...
[1] "Initial PCA loss: 161.299596422154"
Time difference of 23.41366 secs
[1] "nb-PCA loss: 160.982690156888"
Fri Jul 27 23:31:10 2018: Fitting the data ...
Fri Jul 27 23:31:19 2018: P-value calculation ...
Fri Jul 27 23:31:20 2018: Zscore calculation ...
Warning message:
In autoCorrectR(ods, q, theta, ...) :
Fit didn't converge with warning: NEW_X
Should I be concerned about it or it does not influence the results?
Thanks!
Sergey
Dear @naumenko-sa,
this warning comes occasionally when the optim function of the autoCorrect fit did not converged.
You could call each analysis function by your self and pass on the control argument for optim to set the maximal iterations in the autoCorrect call. But mainly this should not influence your results.
Since we are currently improving this package I will add later better error handling and better messages for the user.
I will come back around 2 weeks.
For reference of anyone who comes across this issue, the suggestion by @c-mertes worked for me and the specific formatting of the command was:
ods <- autoCorrect(ods,q=20,control=list(maxit=20000))
But I am also hoping @c-mertes would be willing to expand a little bit about the forthcoming improvements to the software. Is it mostly coding/error handling/etc. improvements or are there major scientific (or bugfix) improvements? I'm just trying to judge how "valid" I should consider results of the current software, which I understand is still in development and published only as a preprint. In my testing, the software does seem to behave reasonably and it caught the outlier that I knew to be true, but I am looking to potentially run follow-up experiments, etc., based on the results and would appreciate any insight into how much we should be trusting the software at this point in time, versus how important it is to wait for the updated release.
Thank you!
Garrett
Dear @WGarrettJenkinson,
thank you for trying out our software and following up this issue. We changed a bit the autoencoder model, which will change the results. We just want to make sure before sending it out that everything works well. The upcoming changes will improve the outlier calls and the normalization. So from the current point, it will rather expand the call set instead of reducing it. So in other words, if you found already something significant it, will probably stay there. If you have 1 or 2 weeks time to wait we should be ready with the new version.
Best,
Christian
That is very helpful information for my planning, and I will be eagerly awaiting the new release. Thank you for your responsiveness and for producing such a useful tool. Have a nice weekend.
Hi Sergey,
hi @WGarrettJenkinson,
as said above, we improved our method and resolved a bug which led to these convergence issues.
We further improved the default value for the used encoding dimension (q). This dimension is similar to the number of principal components in PCA, it used to be 5 in the old version.
To get the optimal dimension run the function findEncodingDim but this will take some time.
To obtain fast results use the new default or the number of samples divided by 5.
Good luck and thanks for trying,
Felix
Hey all, since we did not hear anything from you and we think that this issue should be resolved by the new version of OUTRIDER, I would close the issue in a week. If you still have some issues regarding the convergence, please let us know here in this issue.
The new version does seem to converge for me in all my testing, thank you.
I do have an unrelated question (and can open a new issue if needed): in the new version, how can I get results without applying any auto-encoder/correction? I get an error if I try to skip that step. Can I pass a manual correction that is an identity matrix or some other such "trick"?
Hey @WGarrettJenkinson,
since the normalizationFactors represent the expected mean you can use a rowMeans to run OUTRIDER with no normalization and then calculate all the values manually.
See below an example
library(OUTRIDER)
# default analysis
ods <- makeExampleOutriderDataSet()
ods <- OUTRIDER(ods)
# No normalization (init with gene means)
ods1 <- OutriderDataSet(countData=counts(ods), colData=colData(ods))
normalizationFactors(ods1) <- matrix(rowMeans(counts(ods1)), ncol=ncol(ods1), nrow=nrow(ods1))
# Fit the theta and calculate the pvalues manually
ods1 <- fit(ods1)
ods1 <- computePvalues(ods1)
ods1 <- computeZscores(ods1)
# check results
results(ods)
results(ods1)
which(assay(ods, "trueOutliers") != 0, arr.ind=TRUE)
That is very helpful, thank you! And congratulations on the AJHG publication that I just saw is published.