BF is `NA`

[mattansb]

Revisiting #68 & easystats/bayestestR#627, we're still not getting BFs.

(I'm comparing the same model to itself, so log(BF) should be 0, but getting NA):

library(blavaan)

data("PoliticalDemocracy", package = "lavaan")

model <- "
  # latent variable definitions
  dem60 =~ y1 + a*y2
  dem65 =~ y5 + a*y6

  # regressions
  dem65 ~ dem60

  # residual correlations
  y1 ~~ y5
"

bfit <- blavaan::bsem(model,
                      data = PoliticalDemocracy,
                      n.chains = 1, burnin = 50, sample = 100
)
#> Warning messages:
#> 1: The largest R-hat is NA, indicating chains have not mixed.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#r-hat 
#> 2: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#bulk-ess 
#> 3: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#tail-ess 
#> 4: blavaan WARNING: As specified, the theta covariance matrix is neither diagonal nor unrestricted, so the actual prior might differ from the stated prior. See
#>  https://arxiv.org/abs/2301.08667 

blavaan::blavCompare(bfit, bfit)
#> 
#> WAIC estimates: 
#>  object1:  1436.048 
#>  object2:  1436.048 
#> 
#>  ELPD difference & SE: 
#>     0.000    0.000 
#> 
#> LOO estimates: 
#>  object1:  1435.986 
#>  object2:  1435.986 
#> 
#>  ELPD difference & SE: 
#>     0.000    0.000 
#> 
#> Laplace approximation to the log-Bayes factor
#> (experimental; positive values favor object1):       NA

[ecmerkle]

Thanks for following up, I think this is a different issue than the one described in #68.

I think the issue now is related to the fourth warning message about the theta covariance matrix. The y1 ~~ y5 residual covariance leads to a covariance matrix with one free off-diagonal element and others fixed to 0. In such a case, the prior for the covariance matrix can be influenced by the positive definite constraint, and blavaan cannot necessarily evaluate the correct prior (which throws off the BF calculation) so an NA is reported. For more detail, see the "positive definite constraints" section here.

In your specific case, I think it is a false alarm because there can be one free covariance without worrying about positive definiteness. I can improve the check here. And if you remove the residual covariance, you should get the BF with the current version of blavaan.

[ecmerkle]

You should now be able to obtain the Bayes factor for your model. If you were to add a second residual covariance to your model involving either y1 or y5, then the Bayes factor would again become NA due to the positive definite issue from my first reply. And the Bayes factor is also currently turned off for models that have ordinal observed variables. In general, I am trying to be careful about only returning the Bayes factor when it is clear that we can reliably evaluate the priors in the model.