check_model() does not produce Uniformity of Residuals plot for binomial models
Closed this issue · 9 comments
I was playing with a glm() to check how various packages check for assumptions, and I saw that for binomial models it does not produce the QQ plot (although the documentation says it should).
Reproducible example
data(mtcars)
library(performance)
model <- glm(am ~ wt + cyl, data = mtcars, family = binomial)
check_model(model)
check_residuals(model)
check_models() produces this:
The function for it seems to work, as if you call check_residuals() directly it says:
> check_residuals(model) OK: Simulated residuals appear as uniformly distributed (p = 0.912).
Not sure why there is no plot. Maybe I am making a mistake somewhere.
I think the problem is that the lower left panel is empty.
Same here but with poisson glm and negative binomial glm. However, after having installed qqplotr package it worked!
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22631)
Can you post your session info?
Hi @strengejacke.
here is the current session info, which is after I have installed qqplotr_0.0.6
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22631)
Matrix products: default
oocale:
[1] LC_COLLATE=Portuguese_Brazil.utf8 LC_CTYPE=Portuguese_Brazil.utf8
[3] LC_MONETARY=Portuguese_Brazil.utf8 LC_NUMERIC=C
[5] LC_TIME=Portuguese_Brazil.utf8
time zone: America/Sao_Paulo
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] performance_0.12.2
loaded via a namespace (and not attached):
[1] utf8_1.2.4 generics_0.1.3 robustbase_0.99-3
[4] qqconf_1.3.2 bitops_1.0-8 lattice_0.22-6
[7] pracma_2.4.4 lme4_1.1-35.5 magrittr_2.0.3
[10] caTools_1.18.2 grid_4.4.1 iterators_1.0.14
[13] foreach_1.5.2 doParallel_1.0.17 Matrix_1.7-0
[16] opdisDownsampling_1.0.1 ggrepel_0.9.5 mgcv_1.9-1
[19] fansi_1.0.6 scales_1.3.0 codetools_0.2-20
[22] cli_3.6.3 rlang_1.1.4 pbmcapply_1.5.1
[25] munsell_0.5.1 splines_4.4.1 withr_3.0.1
[28] tools_4.4.1 datawizard_0.12.2 parallel_4.4.1
[31] nloptr_2.1.1 see_0.8.5 minqa_1.2.7
[34] dplyr_1.1.4 colorspace_2.1-1 ggplot2_3.5.1
[37] bayestestR_0.14.0 boot_1.3-30 vctrs_0.6.5
[40] R6_2.5.1 lifecycle_1.0.4 MASS_7.3-61
[43] insight_0.20.2 pkgconfig_2.0.3 pillar_1.9.0
[46] gtable_0.3.5 glue_1.7.0 Rcpp_1.0.13
[49] DEoptimR_1.1-3 tibble_3.2.1 tidyselect_1.2.1
[52] farver_2.1.2 patchwork_1.2.0 nlme_3.1-165
[55] labeling_0.4.3 qqplotr_0.0.6 compiler_4.4.1
[58] twosamples_2.0.1
I discovered this bug yesterday when several of my students were unable to generate a QQ plot for logistic regression models. After replicating the issue myself, I confirmed the problem. I attempted to adjust the options in check_model() and updated both the performance package and all other installed packages, but the issue persisted:
The weird part was that the code didn't produce any warning or error messages, unlike when other dependencies are missing. However, after running simulate_residuals() %>% plot(), the warning message finally appeared:
> simulate_residuals(fit1) %>% plot()
For confidence bands, please install `qqplotr`.
Warning messages:
1: Computation failed in `stat_qq()`.
Caused by error in `distribution()`:
! could not find function "distribution"
2: Computation failed in `stat_qq_line()`.
Caused by error in `distribution()`:
! could not find function "distribution"
And here´s the plot:
Below you can find my session info. However, we tried in several computers at work and get the same problem.
> sessionInfo()
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22631)
Matrix products: default
locale:
[1] LC_COLLATE=Spanish_Argentina.utf8 LC_CTYPE=Spanish_Argentina.utf8
[3] LC_MONETARY=Spanish_Argentina.utf8 LC_NUMERIC=C
[5] LC_TIME=Spanish_Argentina.utf8
time zone: America/Buenos_Aires
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] lubridate_1.9.3 forcats_1.0.0 stringr_1.5.1 dplyr_1.1.4
[5] purrr_1.0.2 readr_2.1.5 tidyr_1.3.1 tibble_3.2.1
[9] ggplot2_3.5.1 tidyverse_2.0.0 janitor_2.2.0 skimr_2.1.5
[13] dlookr_0.6.3 gtsummary_1.7.2 performance_0.12.0
loaded via a namespace (and not attached):
[1] gridExtra_2.3 rlang_1.1.4 magrittr_2.0.3
[4] hrbrthemes_0.8.7 snakecase_0.11.1 compiler_4.4.1
[7] mgcv_1.9-1 systemfonts_1.1.0 vctrs_0.6.5
[10] sysfonts_0.8.9 httpcode_0.3.0 pkgconfig_2.0.3
[13] crayon_1.5.3 fastmap_1.2.0 labeling_0.4.3
[16] utf8_1.2.4 promises_1.3.0 rmarkdown_2.27
[19] tzdb_0.4.0 nloptr_2.1.0 bit_4.0.5
[22] xfun_0.45 showtext_0.9-7 jsonlite_1.8.8
[25] later_1.3.2 parallel_4.4.1 R6_2.5.1
[28] stringi_1.8.4 pagedown_0.20 boot_1.3-30
[31] extrafontdb_1.0 Rcpp_1.0.12 knitr_1.47
[34] DHARMa_0.4.6 base64enc_0.1-3 extrafont_0.19
[37] httpuv_1.6.15 Matrix_1.7-0 splines_4.4.1
[40] timechange_0.3.0 tidyselect_1.2.1 rstudioapi_0.16.0
[43] curl_5.2.1 lattice_0.22-6 shiny_1.8.1.1
[46] withr_3.0.0 bayestestR_0.13.2 evaluate_0.24.0
[49] xml2_1.3.6 pillar_1.9.0 insight_0.20.1
[52] generics_0.1.3 vroom_1.6.5 hms_1.1.3
[55] munsell_0.5.1 scales_1.3.0 minqa_1.2.7
[58] xtable_1.8-4 glue_1.7.0 gdtools_0.3.7
[61] tools_4.4.1 gfonts_0.2.0 see_0.8.4
[64] lme4_1.1-35.4 reactable_0.4.4 grid_4.4.1
[67] Rttf2pt1_1.3.12 datawizard_0.11.0 colorspace_2.1-0
[70] nlme_3.1-164 patchwork_1.2.0 repr_1.1.7
[73] showtextdb_3.0 cli_3.6.3 kableExtra_1.4.0
[76] fontBitstreamVera_0.1.1 fansi_1.0.6 broom.helpers_1.15.0
[79] viridisLite_0.4.2 svglite_2.1.3 gt_0.10.1
[82] gtable_0.3.5 digest_0.6.35 fontquiver_0.2.1
[85] ggrepel_0.9.5 crul_1.4.2 farver_2.1.2
[88] htmlwidgets_1.6.4 htmltools_0.5.8.1 lifecycle_1.0.4
[91] mime_0.12 fontLiberation_0.1.0 bit64_4.0.5
[94] MASS_7.3-60.2
Strange. Do you have a traceback()?
Short remark: issue is probably somewhere here:
see/R/plot.performance_simres.R
Lines 108 to 128 in b294a18
Thanks, should be fixed in #363