coverage
macartan opened this issue · 18 comments
started trying to figure out coverage issues but some confusing results:
file_coverage(source_files = "R/make_par_values.R", test_files = "tests/testthat/test_make_par_values.R")
Test passed 🌈
Test passed 🌈
Test passed 🌈
Coverage: 0.00%
R/make_par_values.R: 0.00%
also: almost all of these produce warnings
#' @examples
#'
#' # the below methods can be applied to either priors or
#' # param_values by specifying the desired option in \code{alter}
#'
#' model <- CausalQueries::make_model("X -> M -> Y; X <-> Y")
#'
#' #altering values using \code{alter_at}
#' CausalQueries:::make_par_values(model = model,
#' x = c(0.5,0.25),
#' alter_at = "node == 'Y' & nodal_type %in% c('00','01') & given == 'X.0'")
#'
#' #altering values using \code{param_names}
#' CausalQueries:::make_par_values(model = model,
#' x = c(0.5,0.25),
#' param_names = c("Y.10_X.0","Y.10_X.1"))
#'
#' #altering values using \code{statement}
#' CausalQueries:::make_par_values(model = model, x = c(0.5,0.25), statement = "Y[M=1] > Y[M=0]")
#'
#' #altering values using a combination of other arguments
#' CausalQueries:::make_par_values(model = model,
#' x = c(0.5,0.25), node = "Y", nodal_type = c("00","01"), given = "X.0")
@macartan I realized why the coverage is so low: covr::package_coverage() excludes all tests excluded from CRAN using testthat::skip_on_cran(). @till-tietz what is the reason for excluding all those tests from CRAN?
Some of those tests have long runtimes which can often trigger unnecessary manual inspection on CRAN. But would be good to go through and check which we can include.
With the hack @macartan suggested, the current coverage is at 91.90%. @till-tietz do you happen to know what the acceptable tests runtime for CRAN is? A quick search did not return anything definite.
> Sys.setenv(NOT_CRAN = "true")
> covr::package_coverage()
CausalQueries Coverage: 91.90%
R/get_event_prob.R: 75.00%
R/get_type_prob.R: 78.12%
R/make_models.R: 79.19%
R/make_par_values.R: 82.32%
R/helpers.R: 87.67%
R/query_model.R: 88.84%
R/realise_outcomes.R: 89.33%
R/parmap.R: 90.32%
src/stanExports_simplexes.h: 90.83%
R/update_model.R: 91.67%
R/get_nodal_types.R: 92.86%
R/plot_dag.R: 94.12%
R/set_confounds.R: 94.12%
R/set_priors.R: 95.24%
R/make_data.R: 96.15%
R/set_parameter_matrix.R: 96.77%
R/set_restrictions.R: 97.11%
R/map_query_to_nodal_type.R: 97.62%
R/prep_stan_data.R: 98.36%
R/clean_params.R: 100.00%
R/data_helpers.R: 100.00%
R/draw_causal_type.R: 100.00%
R/get_ambiguities_matrix.R: 100.00%
R/get_causal_types.R: 100.00%
R/get_parents.R: 100.00%
R/get_query_types.R: 100.00%
R/get_type_helpers.R: 100.00%
R/map_query_to_causal_type.R: 100.00%
R/misc.R: 100.00%
R/query_helpers.R: 100.00%
R/set_parameters.R: 100.00%
R/set_prior_distribution.R: 100.00%
R/simulate_events.R: 100.00%
src/get_type_prob_helpers_arma.cpp: 100.00%
src/realise_outcome_c.cpp: 100.00%
src/stanExports_simplexes.cc: 100.00%
@gerasy1987 I'm not sure if there is official guidance on that but Rcmd checks often return notes when a test or example doesn't complete in under 5 seconds.
It's not a dealbreaker on CRAN but fast tests + examples are preferred to reduce server load as tests are run on multiple platforms + R versions
@macartan @till-tietz @lilymedina Here is the package_coverage() output for the package on the main branch:
> Sys.setenv(NOT_CRAN = "true")
> covr::package_coverage()
CausalQueries Coverage: 91.43%
R/make_models.R: 74.25%
R/make_par_values.R: 77.08%
R/get_type_prob.R: 78.12%
R/query_model.R: 84.68%
R/realise_outcomes.R: 90.80%
src/stanExports_simplexes.h: 90.83%
R/parmap.R: 92.11%
R/get_nodal_types.R: 92.86%
R/update_model.R: 93.02%
R/map_query_to_nodal_type.R: 94.17%
R/set_confounds.R: 95.00%
R/helpers.R: 95.02%
R/set_restrictions.R: 95.59%
R/make_data.R: 95.65%
R/plot_dag.R: 96.08%
R/set_parameter_matrix.R: 96.97%
R/set_priors.R: 97.56%
R/prep_stan_data.R: 98.59%
R/clean_params.R: 100.00%
R/data_helpers.R: 100.00%
R/draw_causal_type.R: 100.00%
R/get_ambiguities_matrix.R: 100.00%
R/get_causal_types.R: 100.00%
R/get_event_prob.R: 100.00%
R/get_parents.R: 100.00%
R/get_query_types.R: 100.00%
R/get_type_helpers.R: 100.00%
R/map_query_to_causal_type.R: 100.00%
R/misc.R: 100.00%
R/query_helpers.R: 100.00%
R/set_parameters.R: 100.00%
R/set_prior_distribution.R: 100.00%
R/simulate_events.R: 100.00%
src/get_type_prob_helpers_arma.cpp: 100.00%
src/realise_outcome_c.cpp: 100.00%
src/stanExports_simplexes.cc: 100.00%
@gerasy1987 @macartan @till-tietz Here's the division of tasks:
| Function | Coverage | Assigned To |
|---|---|---|
| R/make_models.R | 74.25% | Lily |
| R/make_par_values.R | 77.08% | Till |
| R/get_type_prob.R | 78.12% | Gosha |
| R/query_model.R | 84.68% | Macartan |
| R/realise_outcomes.R | 90.80% | Lily |
| src/stanExports_simplexes.h | 90.83% | Till |
| R/parmap.R | 92.11% | Gosha |
| R/get_nodal_types.R | 92.86% | Macartan |
| R/update_model.R | 93.02% | Lily |
| R/map_query_to_nodal_type.R | 94.17% | Till |
| R/set_confounds.R | 95.00% | Gosha |
| R/helpers.R | 95.02% | Macartan |
| R/set_restrictions.R | 95.59% | Lily |
| R/make_data.R | 95.65% | Till |
| R/plot_dag.R | 96.08% | Gosha |
| R/set_parameter_matrix.R | 96.97% | Macartan |
| R/set_priors.R | 97.56% | Lily |
| R/prep_stan_data.R | 98.59% | Till |
@macartan The interactive html report with lines for each function marked is in this zip file
me too
@lilymedina @gerasy1987 @macartan will wrap this up tomorrow plus push some additional fixes
@till-tietz I will push my edits to tests tomorrow and will ping you here when I am done
Updated coverage on the main branch:
> Sys.setenv(NOT_CRAN = "true")
> covr::package_coverage()
CausalQueries Coverage: 95.57%
R/make_par_values.R: 86.71%
src/stanExports_simplexes.h: 90.83%
R/get_nodal_types.R: 92.86%
R/map_query_to_nodal_type.R: 94.17%
R/make_data.R: 95.65%
R/query_model.R: 96.39%
R/make_models.R: 97.11%
R/set_restrictions.R: 98.16%
R/prep_stan_data.R: 98.59%
R/realise_outcomes.R: 98.85%
R/clean_params.R: 100.00%
R/data_helpers.R: 100.00%
R/draw_causal_type.R: 100.00%
R/get_ambiguities_matrix.R: 100.00%
R/get_causal_types.R: 100.00%
R/get_event_prob.R: 100.00%
R/get_parents.R: 100.00%
R/get_query_types.R: 100.00%
R/get_type_helpers.R: 100.00%
R/get_type_prob.R: 100.00%
R/helpers.R: 100.00%
R/map_query_to_causal_type.R: 100.00%
R/misc.R: 100.00%
R/parmap.R: 100.00%
R/plot_dag.R: 100.00%
R/query_helpers.R: 100.00%
R/set_confounds.R: 100.00%
R/set_parameter_matrix.R: 100.00%
R/set_parameters.R: 100.00%
R/set_prior_distribution.R: 100.00%
R/set_priors.R: 100.00%
R/simulate_events.R: 100.00%
R/update_model.R: 100.00%
src/get_type_prob_helpers_arma.cpp: 100.00%
src/realise_outcome_c.cpp: 100.00%
src/stanExports_simplexes.cc: 100.00%
Just pushed the final make_par_values tests. Running final checks on tests and closing this if everything runs smoothly.
@macartan @gerasy1987 @lilymedina coverage looks excellent:
CausalQueries Coverage: 96.62%
src/stanExports_simplexes.h: 90.93%
R/get_nodal_types.R: 92.86%
R/map_query_to_nodal_type.R: 94.17%
R/make_data.R: 95.65%
R/query_model.R: 96.39%
R/make_models.R: 97.09%
R/set_restrictions.R: 98.16%
R/prep_stan_data.R: 98.59%
R/realise_outcomes.R: 98.85%
R/make_par_values.R: 99.34%
R/clean_params.R: 100.00%
R/data_helpers.R: 100.00%
R/draw_causal_type.R: 100.00%
R/get_ambiguities_matrix.R: 100.00%
R/get_causal_types.R: 100.00%
R/get_event_prob.R: 100.00%
R/get_parents.R: 100.00%
R/get_query_types.R: 100.00%
R/get_type_helpers.R: 100.00%
R/get_type_prob.R: 100.00%
R/helpers.R: 100.00%
R/make_events.R: 100.00%
R/map_query_to_causal_type.R: 100.00%
R/misc.R: 100.00%
R/parmap.R: 100.00%
R/plot_dag.R: 100.00%
R/query_helpers.R: 100.00%
R/set_confounds.R: 100.00%
R/set_parameter_matrix.R: 100.00%
R/set_parameters.R: 100.00%
R/set_prior_distribution.R: 100.00%
R/set_priors.R: 100.00%
R/update_model.R: 100.00%
src/get_type_prob_helpers_arma.cpp: 100.00%
src/realise_outcome_c.cpp: 100.00%
src/stanExports_simplexes.cc: 100.00%
Thank you all for your amazing efforts on that front. Closing this issue.