samelamin/spark-bigquery

BQ Streaming Inserts

Closed this issue · 2 comments

Hi Sam,

You mentioned if more people requested streaming inserts for Structured Steaming youd consider implementing it.

What if I commissioned your time to do this when you are feeling better? It does cost, but at 5 cents per GB, for the benefit of lower latency, cutting out GCS and having duplicates handled gracefully in the sink (bit of a problem for me at the moment), this would be an awesome feature. And noone else is doing it for Spark (that I know of at least).

Just a thought, of course just decline if its of no interest, no problems.

Regardless, hope you are feeling better mate and the eye recovering well!

Take it easy
Kurt