Different inferential stats (t vs z) values in medmod module

[richlopez41]

I've run into an issue where I'm getting slightly different results, especially with smaller (e.g., N ≤ 30) sample sizes, when I run moderation analyses with medmod (versus results from the equivalent model specified using a general linear model). Specifically, the Z values (and associated p values) for the parameter estimates from medmod are off a bit when comparing them with the equivalent t (and p) values for parameter estimates.

My understanding is that this arises because the med/mod functions in the medmod module are scaffolded onto lavaan, which runs a Wald test and computes Z-statistics that are tested against a chi-square (vs student's t) distribution. Many times, Wald (and vanilla t) tests will lead you to the same decision/inference (i.e., to reject the null or not). But, it seems that with smaller sample sizes the p-values can be different and that can be problematic.

If possible, could a future release of medmod give the option of computing t (and associated p) values instead of the Wald test (with z and p values) inherited from lavaan?

[raviselker]

So, I've created medmod partly because I wanted to create a package/module that makes it easy for people to use the power of lavaan for these type of analyses and possibly also move on to lavaan for fitting more complicated models. For this use case I think it's important that the results are the same as the results in lavaan. But were also working on a updated version of linear regression in jamovi, which will make it easy to do moderation analysis and get t-values (or you can just use the GAMLj module which already makes it easy).

[richlopez41]

Thanks for your reply! That makes sense (i.e., medmod serving as a stepping stone to specify more complex models in lavaan), and I have been using GAMLj for models with more than just 2 predictors and their interaction (or mediation of X -> M -> Y). I guess I have two followup questions:

1. What do I do in the case of differing t vs. z (and therefore p) values between medmod and GAMLj, say for an interaction term being significant (at alpha=.05) in one set of results but not the other?

2. Also, would it be possible to specify slightly more complex moderation and mediation models in medmod ? The case I have in mind is when you want to control for covariates of non-interest..that is, look at the influence of X1, X2, and their interaction on Y but want to control for covariate W's influence on Y. Similarly, can this be done for mediation (i.e., assess direct effect of X on Y and indirect effect of X on Y via M, while controlling for W's influence on Y)? I've been able to fit these models in lavaan in R and the PROCESS add-on module in SPSS, respectively, but since it's a relatively common analysis I was wondering if it could be implemented in a future release of medmod ?

[juju-bowow]

Hi @raviselker and @richlopez41

I too experienced this same confusion, as have some colleagues, and was fortunate enough to find this well crafted question and answer.

Just my 2c, but it would be very useful if the output of medmod noted its use of Wald tests and lavaan, so as to resolve this potential confusion immediately when it arises, and to ensure that users are not under the misunderstanding that conventional general linear approaches are being used by medmod, in the case they don't compare the output to a general linear model for example. I know that the table clearly includes Z-statistics, but I know that this is still missed or leads to this same question for many users new to modmed and jamovi.

Lastly thank you for providing this highly accessible and useful tool :)