samelamin/spark-bigquery

Table Region not EU

kurtmaile opened this issue · 3 comments

kurtmaile commented

Hi Sam,

My BQ tables created from this library (Create Disposition is CREATE_IF_NEEDED) do not seem to recognise the 'EU' region setting and they all end up with 'Data Location' as 'US' in BQ when looking at the Table details.

spark.sqlContext.setBigQueryDatasetLocation("EU")

Any ideas?

Cheers!

samelamin commented

Hi @kurtmaile that's very strange how are you saving the table? Via streaming or saveAsBigQueryTable?

I'll try to reproduce when I am home.

FYI you do not need the create disposition because it's create if needed by default

Can you make sure the dataset is in the EU location

kurtmaile commented

Thanks for the swift reply.

This is via 'saveAsBigQueryTable'

I just checked and created a new dataset set in EU - then redid the test and the table was in EU so seems the dataset defaulted to US. I would have noticed that except on the original dataset it didnt have the location row visible - whereas in the new one it does, must only show that row if its non US?

Thanks for your help

samelamin commented

No worries glad I can help!