GoogleCloudPlatform/cloud-sql-python-connector

Create an Official Beam I/O Connector for Cloud SQL

FearTheMonstar opened this issue · 2 comments

FearTheMonstar commented

Feature Description

The current Cloud SQL Python connector is great. But in it's current form, when used via Apache Beam/Google Cloud Dataflow, it is not recognized as an official Apache Beam framework construct. This can lead to unbalanced workers in Dataflow when dealing with any kind of volume in a pipeline.

I suggest creating an official Beam I/O Connector so that Cloud SQL can have optimized usage with Beam/Dataflow.

Sample code

No response

Alternatives Considered

There is current non-native Cloud SQL support, but it requires a custom container. This can cause slowdowns and longer approval processes at regulated customers. An official native I/O connector would be ideal.

Additional Details

No response

jackwotherspoon commented

Thanks for raising the issue! We will chat with stakeholders and monitor any upvotes here to help prioritize this potential work.

Related to #1057

  • ❤️2
danvau7 commented

Would like to add 1 upvote to this item :)

  • ❤️1