Stan warnings in 2016 (and 2020)

Question

Stan warnings in 2016 (and 2020)

pqnelson opened this issue 4 years ago · 5 comments

Running the final_2016.R code, stan throws warnings:

Warning messages:
1: There were 1 divergent transitions after warmup. Increasing adapt_delta above 0.8 may help. See
http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup 
2: Examine the pairs() plot to diagnose sampling problems
 
3: The largest R-hat is NA, indicating chains have not mixed.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#r-hat 
4: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#bulk-ess 
5: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#tail-ess

This is with n_iter <- 2500, 4 chains, and rstan::stan_version() tells me I'm running version 2.21.0 (I'm using R 4.0.2 on Windows 10, if that's germane; I'd be happy to supply any other versioning or hardware info).

If I run it with n_iter <- 1000 on 4 chains (as it is by default), I get the last three error messages (largest R-hat is NA, Bulk ESS too low, and Tail ESS too low).

I'd give you the resulting .rds file for the output, but it's several gigs in size.

Answer 1 · 2020-07-10T02:57:09.000Z

Further inspection reveals extract(out, "mu_a[251]") is a vector of zeros, with a NaN Rhat, and a NaN n_eff.

Answer 2 · 2020-07-10T03:39:55.000Z

Good to know, perhaps reassuring.

…

On Jul 9, 2020, 22:57 -0400, Alex ***@***.***>, wrote: Further inspection reveals extract(out, "mu_a[251]") is a vector of zeros, with a NaN Rhat, and a NaN n_eff. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Answer 3 · 2020-07-10T04:31:46.000Z

Not really, if one were to perturb mu_a[current_T] to something like mu_a[current_T] = raw_mu_a[current_T] * sigma_a, one still gets warning messages after running 2016:

Warning messages:
1: The largest R-hat is 1.09, indicating chains have not mixed.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#r-hat 
2: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#tail-ess

Answer 4 · 2020-07-10T05:09:43.000Z

Thank you. I will investigate more.

…

On Jul 10, 2020, 00:31 -0400, Alex ***@***.***>, wrote: Not really, if one were to perturb mu_a[current_T] to something like mu_a[current_T] = raw_mu_a[current_T] * sigma_a, one still gets warning messages after running 2016: Warning messages: 1: The largest R-hat is 1.09, indicating chains have not mixed. Running the chains for more iterations may help. See http://mc-stan.org/misc/warnings.html#r-hat 2: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable. Running the chains for more iterations may help. See http://mc-stan.org/misc/warnings.html#tail-ess — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

Answer 5 · 2020-08-02T21:35:10.000Z

@pqnelson This should be fixed now. The large standard deviation on the national vote parameter was leading to some tough variance estimation. That parameter has been removed from the latest models though and I think you'll get stable model results without convergence errors. Please let me know.