rcshubhadeep/beam-mysql-connector

An io connector of Apache Beam to access MySQL databases.

Beam - MySQL Connector

Beam - MySQL Connector is an io connector of Apache Beam to access MySQL databases.

Installation

pip install beam-mysql-connector

Getting Started

• Read From MySQL

from beam_mysql.connector import splitters
from beam_mysql.connector.io import ReadFromMySQL


read_from_mysql = ReadFromMySQL(
        query="SELECT * FROM test_db.tests;",
        host="localhost",
        database="test_db",
        user="test",
        password="test",
        port=3306,
        splitter=splitters.NoSplitter()  # you can select how to split query from splitters
)

• Write To MySQL

from beam_mysql.connector.io import WriteToMySQL


write_to_mysql = WriteToMySQL(
        host="localhost",
        database="test_db",
        table="tests",
        user="test",
        password="test",
        port=3306,
        batch_size=1000,
)

License

MIT License. Please refer to the LICENSE.txt, for further details.