easystats/parameters

Inconsistency in p_adjust when working with emmeans/emGrid objects

Generalized opened this issue · 0 comments

Let's assume I have an emmeans/emGrid object already created:

> emg
 Week    emmean    SE  df asymp.LCL asymp.UCL
 Week 1  -0.317 0.132 Inf    -0.577   -0.0573
 Week 2  -0.633 0.136 Inf    -0.900   -0.3656
 Week 4  -0.719 0.178 Inf    -1.068   -0.3696
 Week 8  -0.984 0.166 Inf    -1.309   -0.6587
 Week 16 -1.198 0.162 Inf    -1.514   -0.8810
 Week 56 -1.456 0.290 Inf    -2.025   -0.8879

Covariance estimate used: vbeta 
Confidence level used: 0.95 

Now I want both CI + p-value to be adjusted simultaneously using the mutivariate t distribution:

> set.seed(1000); update(emg, infer = c(TRUE, TRUE), adjust="mvt")
 Week    emmean    SE  df asymp.LCL asymp.UCL z.ratio p.value
 Week 1  -0.317 0.132 Inf    -0.652    0.0181  -2.393  0.0717
 Week 2  -0.633 0.136 Inf    -0.977   -0.2880  -4.643  <.0001
 Week 4  -0.719 0.178 Inf    -1.170   -0.2681  -4.033  0.0003
 Week 8  -0.984 0.166 Inf    -1.404   -0.5642  -5.929  <.0001
 Week 16 -1.198 0.162 Inf    -1.606   -0.7889  -7.413  <.0001
 Week 56 -1.456 0.290 Inf    -2.190   -0.7227  -5.021  <.0001

Covariance estimate used: vbeta 
Confidence level used: 0.95 
Conf-level adjustment: mvt method for 6 estimates 
P value adjustment: mvt method for 6 tests 

Now I'm trying to do the same with parameters (expecting it will pass the parameter to emmeans since it understands this class):

> parameters(emg, p_adjust = "mvt")
Week    | Marginal Means |   SE |         95% CI |     z |      p
-----------------------------------------------------------------
Week 1  |          -0.32 | 0.13 | [-0.58, -0.06] | -2.39 | 0.017 
Week 2  |          -0.63 | 0.14 | [-0.90, -0.37] | -4.64 | < .001
Week 4  |          -0.72 | 0.18 | [-1.07, -0.37] | -4.03 | < .001
Week 8  |          -0.98 | 0.17 | [-1.31, -0.66] | -5.93 | < .001
Week 16 |          -1.20 | 0.16 | [-1.51, -0.88] | -7.41 | < .001
Week 56 |          -1.46 | 0.29 | [-2.02, -0.89] | -5.02 | < .001

p-value adjustment method: mvt
Warning message:
`p_adjust` must be one of holm, hochberg, hommel, bonferroni, bh, by, fdr, none, tukey, scheffe, sidak 

OK, let's try "bh"

> parameters(emg, p_adjust = "bh")
Error in match.arg(method) : 
  'arg' should be one of “holm”, “hochberg”, “hommel”, “bonferroni”, “BH”, “BY”, “fdr”, “none”

Also doesn't work. So maybe the "BH":

> parameters(emg, p_adjust = "BH")
Week    | Marginal Means |   SE |         95% CI |     z |      p
-----------------------------------------------------------------
Week 1  |          -0.32 | 0.13 | [-0.58, -0.06] | -2.39 | 0.017 
Week 2  |          -0.63 | 0.14 | [-0.90, -0.37] | -4.64 | < .001
Week 4  |          -0.72 | 0.18 | [-1.07, -0.37] | -4.03 | < .001
Week 8  |          -0.98 | 0.17 | [-1.31, -0.66] | -5.93 | < .001
Week 16 |          -1.20 | 0.16 | [-1.51, -0.88] | -7.41 | < .001
Week 56 |          -1.46 | 0.29 | [-2.02, -0.89] | -5.02 | < .001

p-value adjustment method: Benjamini & Hochberg (1995)

This one worked.

So, to summarize:

  1. the "mvt" isn't passed to emmeans, even if parameters understand this class (but maybe it was not the goal - then I'm sorry for reporting it)
  2. but even if we pick the other options, the hint is given using all small letters