ropensci/targets

Debug/error message upon pipeline error will in some cases refer to wrong target

Closed this issue · 1 comments

Prework

  • I understand and agree to help guide.
  • I understand and agree to contributing guide.
  • New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.

Proposal

Hi Will!

As requested, I managed to create a minimal reprex for the issue with wrong/confusing error/debug messages mentioned in #1340. Essentially, it seems that the logic implemented for getting the relevant target in utils_callr.R:
name <- meta$name[meta$time == max(meta$time, na.rm = TRUE)]
is too simple, because a target that has previously errored will not update its time if erroring again (even also in the event that the code has changed). Therefore, if another target B has been run after the initial error, error messages from the initial target A will now wrongly refer to B, being the latest entry in tar_meta(). I suppose the real fix is ensuring that erroring targets are always logged with updated timestamps (unless this might have unforeseen consequences of course). This will also be very helpful when using the tar_metadata for debugging in general I think, which can sometimes be a bit confusing, with possibly other, previously fixed errors on top.

See reprex below:

library(targets)
list(
  tar_target(A, stop())
)

yields correct target name:

── Debug target A───────
tar_meta(A)
tar_workspace(A)

New target name:

library(targets)
list(
  tar_target(B, stop())
)

yields correct target name as well.

Debug target B ──────────────────────────
tar_meta(B)$error
tar_workspace(B)

Now back to first target name:

library(targets)
list(
  tar_target(A, stop())
)

now yields wrong previous target name (because tar_meta() has not updated the timestamp for A)

── Debug target B ─────────────────────────
tar_meta(B)$error
tar_workspace(B)

Thanks for the reprex and for figuring out why the messages are inconsistent.

Other components depend on the time stamps, and the order of the rows in the metadata file is not reliable, so I think the best solution is to remove the tailored debugging message (10a67af) to avoid creating unmet expectations.