SampleCode For GoogleCloudDataflow This Sample inserts data from PubSubMessage To BigQuery. It is partitioned by day (tableName_YYYYMMdd) and Inserted.
this project managed by maven. if you need change profile to switched configuration,such as tableName. Please switch profiles with maven. default profile is dev.
- see pom.xml and write your gcp configuration to here.
<properties>
<target.project>YOUR_PROJECT_NAME</target.project>
<target.subscription>YOUR_SUBSCRIPTION_PATH</target.subscription>
<target.bigquery.table>PROJECT_NAME:DATASET_NAME.TABLE_NAME</target.bigquery.table>
<target.templocation>YOUR_GCS_LOCATION FOR TEMP(example: gs://hoge/tmp) </target.templocation>
<target.jobname>YOUR_DATAFLOW_JOBNAME</target.jobname>
<target.staginglocation>YOUR_GCS_LOCATION FOR STAGING (example: gs://hoge/staging)</target.staginglocation>
</properties>