Unable to read BigQuery authorized view
blessymoses opened this issue · 4 comments
I'm reading from a BigQuery Authorized View like this:
beam.io.Read(beam.io.BigQuerySource(query = "SELECT col1 FROM proj2.dataset2.auth_view1", use_standard_sql=True))
I get the following error:
Error: Message: Access Denied: Dataset proj1:dataset1: The user xxxxxx-compute@developer.gserviceaccount.com does not have bigquery.datasets.get permission for dataset proj1:dataset1.
proj1:dataset1 has the base table for the view auth_view1 and the service account has access to the authorized view.
According to this issue in DataflowJavaSDK, dataflow seems to be directly executing some metadata query against the underlying table.
Is there a resolution for this issue?
CC: @chamikaramj
anyone ever get this resolved?
To follow up on @pselden 's ping, this occurred during a TFX pipeline run. I am to understand that the BigQueryExampleGen component does a query against the ostensible query result for purposes of collecting metadata/typing.
Is it possible that this pathway somehow strips away or replaces the target table (which is a view) with the underlying table?
We moved to Apache Beam!
Google Cloud Dataflow for Python is now Apache Beam Python SDK and the code development moved to the Apache Beam repo.
If you want to contribute to the project (please do!) use this Apache Beam contributor's guide. Closing out this issue accordingly.