Add option for SMB joins to fall back to regular joins
clairemcginty opened this issue · 0 comments
clairemcginty commented
Add an option, either pipeline option or API method param, to instruct Scio to fall back to a non-SMB join if it can't perform an SMB join (due to incompatible hash types, missing metadata, etc).
SMB does all this validation at job submission time -- after sc.run() but before the job is submitted to dataflow -- so we should be able to catch exceptions at that time.