Cloud Dataflow Template
kpr6 opened this issue · 3 comments
I made a dataflow template to load .csv files from bucket to BigQuery tables using TemplatingDataflowPipelineRunner, the first time when I run a template job after it's creation, it works fine and loads data to respective location. But when I re-use the template to run another job to migrate another file, the job succeeds but data isn't moved which is strange. Is it that template can only be used once?
Batch jobs like the one you describe and BigQueryIO do not (yet) play well together. See #550
The same issue exists when you read data from BigQuery and write to Pub/Sub
Hi All,
+1
I am sending data from bigquery to datastore. It works fine for the first time then for each execution it is successfull on UI , but no process is actually done.