Add support for skipping a transform if input hasn't changed
Closed this issue · 1 comments
kevinschaper commented
It would be great to not re-transform files that haven't changed. We should use a checksum on the input files that sits in the output location and compare against it to skip the transform step.
I think rather than adding "--force", the least surprising behavior would be to make the checksum check the behavior that is triggered by a flag. I think it's the least surprising behavior for a developer that if you run a transform, a transform happens.
I don't have any great ideas for a name, --skip-on-checksum
?
kevinschaper commented
We ultimately didn't consider this Koza's problem, and left it to the orchestration that calls koza to decide whether or not to run the transform. Closing.