add an "engine" parameter and context property to support bigframes DataFrame output
Closed this issue · 2 comments
tswast commented
Sub-tasks:
- Add a
bigquery_magics.context.engine
option which defaults to "pandas" but can be set to "bigframes" for %%bigquery to return a BigQuery DataFrames object. - The default option can also be overridden by specifying
--engine="bigframes"
in the first line of the %%bigquery magics.
Googlers, see go/bf-magics design.
tswast commented
Benefits for why the user might want to support this:
- Support queries that return > 10 GB of results.
- Hopefully some day: more natural "async" support so as not to block the current thread while the query runs.
tswast commented
Another thought: We should use a code path that doesn't immediately run the query at read_gbq
/ read_gbq_query
time (internal issue 347050555). Otherwise the magic would end up running at least two queries: the original query & one or two to visualize the results from __repr__
.