spotify/scio

Add option for SMB joins to fall back to regular joins

clairemcginty opened this issue · 0 comments

Add an option, either pipeline option or API method param, to instruct Scio to fall back to a non-SMB join if it can't perform an SMB join (due to incompatible hash types, missing metadata, etc).

SMB does all this validation at job submission time -- after sc.run() but before the job is submitted to dataflow -- so we should be able to catch exceptions at that time.