Understanding the effect's direction through the "eff_size" function

[JulianGaviriaL]

The data:
r_001.csv

# The Linear Model:
require(lmerTest)
require(emmeans)
m001 <- lm(Observations ~ Group ,data=r_001)
summary<-summary(m001)
summary

Call:
lm(formula = Observations ~ Group, data = r_001)
Residuals:
Min 1Q Median 3Q Max
-0.22600 -0.08271 -0.03100 0.06900 0.36400

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.38600 0.01978 19.514 <2e-16 ***
Grouppre -0.06439 0.03097 -2.079 0.0411 *

#Compute the effect size
E_r001 <- emmeans(m001, "Group")
ES_r001<- eff_size(E_r001, sigma = sigma(m001), edf = Inf)
ES_r001

contrast effect.size SE df lower.CL upper.CL
ctr - pre 0.485 0.233 74 0.0202 0.95

sigma used for effect sizes: 0.1327
Confidence level used: 0.95

I need to understand the direction of the effect in terms of the difference between levels "ctr" and "pre" of the factor "Group". Why do I obtain a positive effect size (i.e., effect size=0.485) if the difference between the intercept (Groupctr=0.38600) and the slope Grouppre is negative (-0.06439)?

Considering how the data is modeled in lmer ("ctr" is the first level, and "pre" the 2nd level of the "Group" factor), I would expect a negative effect size. Therefore, I would interpret the results as follows: The mean difference between "ctr" and "pre" indicates a larger effect in "ctr". However, I am doubtful due to the positive effect size.

Many thanks in advance for your comments.

[rvlenth]

As you state, the reference level is ctr and the other level is pre, meaning that the regression coefficient for the intercept is the prediction for ctr, and that for Grouppre is the change from ctr, i.e., pre - ctr; and it is negative. However, in the eff_size output, the row label is ctr - pre (not pre - ctr), which therefore must be positive. So your effect size does have the correct sign.

If you want the difference to go the other way, use

eff_size(E_r001, sigma = sigma(m001), edf = Inf, method = "revpairwise")

[JulianGaviriaL]

Thank you very much Prof. @rvlenth

Just one last thing, if possible. How can I list the same effect size from multiple emmGrid objects (>100)? For instance:

ES_r001<- eff_size(E_r001, sigma = sigma(m001), edf = Inf)
ES_r002<- eff_size(E_r002, sigma = sigma(m002), edf = Inf)
....
ES_r200<- eff_size(E_r200, sigma = sigma(m200), edf = Inf)

Ideally, the list would include all effect sizes. Something like:

**#Ideal output** 
    ES          contrast       effect.size      SE      df    lower.CL    upper. CL
ES_r001       ctr - pre         0.299         0.302     70     -0.304       0.901
ES_r002       ctr - pre         -0.03         0.224     70     -0.304       0.044
....
ES_r200       ctr - pre         0.299         0.802     70     -0.284       0.901

The following code with the foreach package did not work

require(foreach)

output_position <- grep("ES",ls())
all_effects <- foreach(i=output_position,.combine="rbind") %do% {
  output <- get(ls()[i])
  unlist(output["ctr-pre",])
}
rownames(all_effects) <- ls()[output_position]
all_effects
b<-as.data.frame(all_effects)

Many thanks in advance

[rvlenth]

You can combine them into a data frame. Starting with your output_position vector

summList <- lapply(output_position, \(x) data.frame(get(x)))
all_ES <- cbind(ES = output_position, do.call(rbind, summList))

(I didn't need the foreach package)

You can do a similar thing to combine them into an emmGrid object by omitting the data.frame() call; however, that requires all of the results to have been obtained from the same model, which I doubt is the case.

[JulianGaviriaL]

thanks @rvlenth
What is "x" in summList?

#the models:
m001 <- lm(Observations ~ Group ,data=r_001)
m002 <- lm(Observations ~ Group ,data=r_002)
m200 <- lm(Observations ~ Group ,data=r_200)

#Compute the effect size
E_r001 <- emmeans(m001, "Group")
E_r002 <- emmeans(m002, "Group")
E_r200 <- emmeans(m200, "Group")
ES_r001<- eff_size(E_r001, sigma = sigma(m001), edf = Inf)
ES_r002<- eff_size(E_r002, sigma = sigma(m002), edf = Inf)
ES_r200<- eff_size(E_r200, sigma = sigma(m200), edf = Inf)

#list the effect size of the contrast ctr - pre from all the emmGrid objects (i.e., ES_rxxx)
output_position <- grep("ES",ls())
summList <- lapply(output_position, \(x) data.frame(get(x)))

Error in get(x) : invalid first argument

[rvlenth]

I misread your code. Instead of using grep, use ls

ESs <- ls(pat = "ES")   # names instead of numbers
summList <- lapply(ES, \(x) data.frame(get(x)))
all_ES <- cbind(ES = ESs, do.call(rbind, summList))

[JulianGaviriaL]

Dear Professor @rvlenth,

Your suggestion works very well. Thank you very much and I wish you happy holidays.

[rvlenth]

Thanks, and happy holidays to you as well.