GoogleCloudDataproc/spark-bigquery-connector
BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
JavaApache-2.0
Issues
- 3
- 2
Add support for BigQuery tables with Collation
#1279 opened by jster1357 - 11
BigQuery Pushdown filtering on Spark 3.4.2
#1207 opened by sid-habu - 1
Issue with Map Type support (pyspark)
#1278 opened by violetautumn - 7
Reading a view results in a empty data frame
#1266 opened by tom-s-powell - 2
`bigquery.tables.create` Permission required on appending data to a BigQuery Table
#1277 opened by smic-datalabs-jdcastro - 0
cloud cost when using spark bq connector
#1274 opened by kaiseu - 2
Disable Schema Changes
#1244 opened by gmiano - 1
- 4
proxyAddress variable is not used if default application credentials method is used
#1260 opened by milicevica23 - 2
"ClusteredFields" creation not supported in Direct write method from Pyspark to BigQuery
#1263 opened by deena10545 - 1
Schema mismatch error needs to be more verbose
#1175 opened by vishalkarve15 - 3
- 1
Request to Stop Enforcing Delete Permissions on Materialization Dataset
#1224 opened by reynoldspravindev - 0
Empty array filtering issue
#1268 opened by vokuk-walkme - 2
Unable to write decimal values with scale > 9 to bigquery using dataproc and spark bigquery connector.
#1257 opened by abhilash499 - 2
NullPointerException - Error creating destination table using the following query
#1256 opened by srikanthnagidi - 2
Failed to write into bigquery using spark 3.2.1
#1264 opened by krishna00000785 - 4
Big Query Full Table Read - Some Partitions take longer bqScanTime as compared to other partitions with similar number of records
#1262 opened by josepaul1988 - 4
Viewing connector metrics
#1252 opened by Swebask - 3
- 1
BigQuery Storage API always returning 200 partitions
#1237 opened by rcanzanese - 1
Permission bigquery.tables.create required on already materialised view
#1246 opened by k-charalampous - 9
How to handle column datatype change?
#1210 opened by nandinir-db - 6
Unable to write to a partitioned table in spark 3.5 with connector version spark-3.4-bigquery-0.34.0
#1228 opened by hierr - 1
- 6
BigNumeric precision is too wide (76), Spark can only handle decimal types with max precision of 38
#1234 opened by John15321 - 2
Indirect write to existing datetime column not possible
#1232 opened by cheare - 2
TIMESTAMP appears to add an hour compared to DATETIME
#1229 opened by Naboo22 - 3
PySpark virtualenv: Missing dependencies lead to No FileSystem for scheme "gs"
#1226 opened by GuiBou165 - 0
error while writing with many timestamp data
#1223 opened by iqbalaey - 2
BIGNUMERIC Precision Handling: Inaccurate Decimal Values
#1208 opened by nabeelq - 4
java.lang.NoSuchMethodError: 'org.apache.spark.sql.catalyst.encoders.ExpressionEncoder org.apache.spark.sql.catalyst.encoders.RowEncoder.apply(org.apache.spark.sql.types.StructType)'
#1209 opened by CatalinMihaiIonescu - 1
Predicate pushdown doesn't work with DateTime BQ field - Spark 3.5, connector version 0.37
#1215 opened by yunus - 2
Error: This connector was made for Scala null, it was not meant to run on Scala 2.12
#1214 opened by malhomaid - 1
Table expiration with write() operation
#1189 opened by mikeperello-scopely - 5
Storage Read API logging
#1204 opened by kristopherkane - 0
Add `createTable(TableInfo)` method to `BigQueryClient`
#1213 opened by jphalip - 1
Map type with Complex Value not supported any more
#1195 opened by walmaaoui - 1
Load failure caused by comment at top of query string (llegalArgumentException: Invalid Table ID)
#1182 opened by bonartm - 0
Idempotent write support in BQ
#1199 opened by SubhamSinghal - 0
Automatically read JSON types
#1203 opened by Guilherme-B - 5
- 1
JARs marked 'latest' not being updated
#1202 opened by dmedora - 5
- 1
Impersonate Service Account
#1192 opened by lcaggio - 3
Unable to overwrite partition
#1184 opened by DevalNaik - 3
AWS Glue - Indirect write mode errors
#1174 opened by adaskaval - 1
Map column of a complex type in values causes error "Data type not expected: struct<...>"
#1185 opened by walmaaoui - 3