ShardingSphere - Distributed Database Ecosphere

Official website: https://shardingsphere.apache.org/

Stargazers over time

Contributor over time

Document

Overview

Apache ShardingSphere is an open-source ecosystem consisted of a set of distributed database solutions, including 3 independent products, JDBC, Proxy & Sidecar (Planning). They all provide functions of data scale-out, distributed transaction and distributed governance, applicable in a variety of situations such as Java isomorphism, heterogeneous language and cloud-native.

Apache ShardingSphere aiming at reasonably making full use of the computation and storage capacity of existed database in a distributed system, rather than a new database. As the cornerstone of enterprises, the relational database still takes a huge market share. Therefore, we prefer to focus on its increment instead of a total overturn.

Apache ShardingSphere begins to focus on pluggable architecture from version 5.x, features can be embedded into project flexibility. Currently, the features such as data sharding, replica query, data encrypt, shadow test, and SQL dialects/database protocols such as MySQL, PostgreSQL, SQLServer, Oracle supported are all weaved by plugins. Developers can customize their ShardingSphere systems just like building LEGO blocks. There are lots of SPI extensions for Apache ShardingSphere now and they are increasing continuously.

ShardingSphere became an Apache Top-Level Project on April 16, 2020.

You are welcome to communicate with the community via the mailing list.

ShardingSphere-JDBC

ShardingSphere-JDBC defines itself as a lightweight Java framework that provides extra services at the Java JDBC layer. With the client end connecting directly to the database, it provides services in the form of a jar and requires no extra deployment and dependence. It can be considered as an enhanced JDBC driver, which is fully compatible with JDBC and all kinds of ORM frameworks.

Applicable in any ORM framework based on JDBC, such as JPA, Hibernate, Mybatis, Spring JDBC Template or direct use of JDBC.
Support any third-party database connection pool, such as DBCP, C3P0, BoneCP, Druid, HikariCP.
Support any kind of JDBC standard database: MySQL, Oracle, SQLServer, PostgreSQL and any SQL92 followed database.

ShardingSphere-Proxy

ShardingSphere-Proxy defines itself as a transparent database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages. Friendlier to DBA, the proxy of MySQL/PostgreSQL version provided now can use any kind of client access (such as MySQL Command Client, MySQL Workbench, Navicat etc.) that is compatible of MySQL/PostgreSQL protocol to operate data.

Transparent to applications, it can be used directly as MySQL and PostgreSQL servers.
Applicable to any kind of compatible client end that is compatible with MySQL and PostgreSQL protocol.

ShardingSphere-Sidecar(TODO)

ShardingSphere-Sidecar (TODO) defines itself as a cloud-native database agent of the Kubernetes environment, in charge of all the access to the database in the form of a sidecar. It provides a mesh layer interacting with the database, we call this Database Mesh.

Database Mesh emphasizes how to connect distributed data-access applications with the databases. Focusing on interaction, it effectively organizes the interaction between messy applications and databases. The applications and databases that use Database Mesh to visit databases will form a large grid system, where they just need to be put into the right positions accordingly. They are all governed by the mesh layer.

	ShardingSphere-JDBC	ShardingSphere-Proxy	ShardingSphere-Sidecar
Database	Any	MySQL/PostgreSQL	MySQL/PostgreSQL
Connections Count Cost	High	Low	High
Supported Languages	Java Only	Any	Any
Performance	Low loss	Relatively High loss	Low loss
Decentralization	Yes	No	No
Static Entry	No	Yes	No

Hybrid Architecture

ShardingSphere-JDBC adopts decentralized architecture, applicable to high-performance light-weight OLTP applications developed with Java; ShardingSphere-Proxy provides static entry and all languages support, applicable for OLAP applications and the sharding databases management and operation situation.

ShardingSphere is an ecosphere consisted of multiple endpoints together. Through a mixed use of ShardingSphere-JDBC and ShardingSphere-Proxy and unified sharding strategy by the same registry center, ShardingSphere can build an application system applicable to all kinds of scenarios. Architects can adjust the system architecture to the most applicable one to current business more freely.

Features

Data Sharding

Database sharding & Table sharding
Replica query
Sharding strategy customization
Centre-less Distributed primary key

Distributed Transaction

Unified Transaction API
XA transaction
BASE transaction

Database Governance

Distributed Governance
Data migration & Scale-out
Observability(Tracing/Metrics) Supported
Data Encryption&Decryption
Shadow Table for Performance Testing

How to Build

Build Apache ShardingSphere

./mvnw clean install -Prelease

Artifact:

shardingsphere-distribution/shardingsphere-src-distribution/target/apache-shardingsphere-${latest.release.version}-src.zip  # Source code package of Apache ShardingSphere
shardingsphere-distribution/shardingsphere-jdbc-distribution/target/apache-shardingsphere-${latest.release.version}-shardingsphere-jdbc-bin.tar.gz  # Binary package of ShardingSphere-JDBC
shardingsphere-distribution/shardingsphere-proxy-distribution/target/apache-shardingsphere-${latest.release.version}-shardingsphere-proxy-bin.tar.gz  # Binary package of ShardingSphere-Proxy
shardingsphere-distribution/shardingsphere-scaling-distribution/target/apache-shardingsphere-${latest.release.version}-shardingsphere-scaling-bin.tar.gz  # Binary package of ShardingSphere-Scaling

Build ShardingSphere-UI

git clone https://github.com/apache/shardingsphere-ui
cd shardingsphere-ui
./mvnw clean install -Prelease