find_best_model issues

[HugoNjb]

Dear Dominique,
Thank you very much for your package, I think it will be very useful to me in the future.
I am notably interested in the find_best_model function and I encontered the following bugs while exploring it.

1

find_best_model.stanreg gives the following error message when a random intercept is entered in the formula :

df <- psycho::emotion
fit <- stan_lmer(Autobiographical_Link ~ Emotion_Condition + Subjective_Valence + (1|Participant_ID), data=df)
best <- find_best_model(fit)

Error in f_i(data_i = data[i, , drop = FALSE], draws = draws, ...) :
unused argument (k_treshold = k_treshold)

Solution
It seems to come from the loo function of the rstanarm package. I added the following lines into find_best_model.stanreg to temporarily solve the problem :

    if (!is.null(k_treshold)) {
      loo <- rstanarm::loo(newfit, k_treshold = k_treshold)
    } else {
      loo <- rstanarm::loo(newfit)
    }

2

Inside find_best_model.stanreg, warning messages are sent when accessing the loos estimates :

loo$elpd_loo

Warning message:
Accessing elpd_loo using '$' is deprecated and will be removed in a future release. Please extract the elpd_loo estimate from the 'estimates' component instead.

Solution
I replaced them with those lines:

    Estimates <- loo[["estimates"]]
    model <- data.frame(
      formula = formula,
      complexity = complexity - 1,
      R2 = R2s[[formula]],
      looic = Estimates["looic","Estimate"],
      looic_se = Estimates["looic","SE"],
      elpd_loo = Estimates["elpd_loo","Estimate"],
      elpd_loo_se = Estimates["elpd_loo","SE"],
      p_loo = Estimates["p_loo","Estimate"],
      p_loo_se = Estimates["p_loo","SE"],
      elpd_kfold = Estimates["p_loo","Estimate"],
      elpd_kfold_se = Estimates["p_loo","SE"]
    )

3

The find_best_model works for merModLmerTest class used in the lmerTest v2.0-36 package but not for lmerModLmerTest class used in lmerTest v3.0.

df <- psycho::emotion
fit2 <- lmerTest::lmer(Autobiographical_Link ~ Emotion_Condition + Subjective_Valence + (1|Participant_ID), data=df)
best <- find_best_model(fit2)

Error in UseMethod("find_best_model") :
no applicable method for 'find_best_model' applied to an object of class "c('lmerModLmerTest', 'lmerMod', 'merMod')"

Solution
From what I understood, the problem seems to come from the find_combinations.formula function. However, I don't know how to simply resolve this problem.

I am new to github and I will try to push my find_best_model.stanreg enhancement in the dev branch immediately.

Thanks in advance !

[DominiqueMakowski]

@HugoNjb Awesome! Let me know if you have any trouble with github 😄

I'll look into the third issue

Thanks for your help!

[HugoNjb]

I sadly didn't find a way to push the script. Could you help me on it ?
This way I will be able to more easily contribute to this project.

You're welcome ! 👍

[DominiqueMakowski]

@HugoNjb Github is a total headache at the beginning, but it's worth to persevere.

I believe there are ways different of doing this, but here's one possibility:

1. Fork this repo (there is a fork button next to the "star" and "follow" buttons), it will create a copy of this package linked to your account.
2. Download the github desktop version and log in
3. Clone your fork on your computer (copy it from github to your computer)
4. In desktop github, create a new branch, called for instance "Fix-BestModel" and publish this branch (it will add it to your version of the repo)
5. Do the changes to the files in your local computer
6. Commit the changes to your branch
7. Create a pull request to merge this branch into the "parent" repo into the master branch

Hope that's helpful ^^

[HugoNjb]

I think I did it right !
Thank you for the explanation !

[DominiqueMakowski]

@HugoNjb I don't have any PR here right now, I believe you only created the branch in your fork. Try going here and creating a pull request from HugoNjb/psycho.R/Fix-BestModel (i.e., your branch) to neuropsychology/psycho.R/dev (comitting to the dev branch would be better than to the master which is supposed to be stable)

[DominiqueMakowski]

Hey, thanks again for your commit. If you install the dev branch devtools::install_github("neuropsychology/psycho.R@dev"), your issues should be gone.

Let me know!

[HugoNjb]

I tested it, and everything seems to work just fine.

Thanks again for the great responses and reactivity !