ropensci/targets

Don't re-run errors

multimeric opened this issue · 2 comments

Prework

  • I understand and agree to help guide.
  • I understand and agree to contributing guide.
  • New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

As explained here, errored targets are always re-run, no matter what. However, there are some cases where a user doesn't want that behaviour. They might have one exceptional branch that has corrupt data where it doesn't make sense to re-run. In this scenario, re-running the target and having it fail each time is a waste of resources. I would like to request some combination of parameters to tar_option_set involving cue and error, that allows errored branches to be skipped until some other cue such as the dependencies or the command changes.

One suggestion is adding: tar_cue(error = FALSE), which would mean "ignore the error status when cueing".

You can accomplish the same thing with a combination of tar_option_set(error = "null") and tar_make(shortcut = TRUE). Example:

library(targets)
tar_option_set(error = "null")
list(
  tar_target(x, stop("abc")),
  tar_target(y, x)
)
tar_make(any_of("x"))
#> ▶ dispatched target x
#> ✖ errored target x
#> ✖ errored pipeline [0.05 seconds]

shortcut = TRUE tells tar_make() to only run the targets you select in the names argument and not traverse the targets upstream.

tar_make(y, shortcut = TRUE)
#> ▶ dispatched target y
#> ● completed target y [0 seconds]
#> ▶ ended pipeline [0.062 seconds]

Thanks, this is a good workaround. However I still wonder if this might be more user friendly as a special cue, because then it could be formalised into the pipeline, and it allows other types of cues like command changes to still trigger re-runs.