large difference across chains
Closed this issue · 2 comments
I am seeing major discrepancies across chains within a run, and I'm not sure what the cause is. I've attached the results from my K=3 spatial run (4 chains, 1000 iterations). The results from chain one show no discrete groups which is directly contrasted by the later 3 chains. How do I know which chain is correct?
I am also repeatedly getting Rhat and ESS errors (see below). I thought these fit the parameters for being accceptably ignored, but perhaps not with such wild divergence of ancestry estimation.
"
1: There were 44 divergent transitions after warmup. See
https://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
to find out why this is a problem and how to eliminate them.
2: There were 22 transitions after warmup that exceeded the maximum treedepth. Increase max_treedepth above 10. See
https://mc-stan.org/misc/warnings.html#maximum-treedepth-exceeded
3: Examine the pairs() plot to diagnose sampling problems
4: The largest R-hat is 1.89, indicating chains have not mixed.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#r-hat
5: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#bulk-ess
6: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#tail-ess
K3_layer.cov.curves.pdf
K3_model.fit.CIs.pdf
K3_structurePlots.pdf
K3_trace.plots.pdf
The first run definitely seems different from the other 3, but it's difficult to visually compare the rest because it doesn't seem like you've dealt with label-switching between independent runs. Check out the visualize-results vignette for more information there.
Re: dealing with warnings - you can pass different stan control parameters (like max_treedepth or adapt_delta) to your conStruct run via the control argument (see here).
Re: choosing which run - looking at the trace plots of the posterior probability and the different parameter estimates for the different runs, it seems like runs 1 and 2 did not have good mixing (note how the posterior probability is still climbing for at least the first 20-40% of the post-burnin iterations). Runs 3&4 look much better (traceplots look like fuzzy caterpillars, etc.). If you want to pick one chain to report, you could pick the one with the highest mean posterior probability (which looks like run 3 in this case), but ideally you'd see that multiple independent runs were converging on the same stationary distribution.
This was super helpful. Running the model longer made no difference in minimizing the discrepancy (actually the opposite), so maybe altering the stan control parameters will help!