monarch-initiative/koza

Add support for skipping a transform if input hasn't changed

Closed this issue · 1 comments

It would be great to not re-transform files that haven't changed. We should use a checksum on the input files that sits in the output location and compare against it to skip the transform step.

I think rather than adding "--force", the least surprising behavior would be to make the checksum check the behavior that is triggered by a flag. I think it's the least surprising behavior for a developer that if you run a transform, a transform happens.

I don't have any great ideas for a name, --skip-on-checksum?

We ultimately didn't consider this Koza's problem, and left it to the orchestration that calls koza to decide whether or not to run the transform. Closing.