Spring Data YugabyteDB (SDYB): Tracking Issue
nchandrappa opened this issue · 3 comments
Motivation
Spring Data modules are one of the widely used Java frameworks for accessing the databases. Spring data provides consistent and familiar APIs for querying the data by supporting Data repository, template methods. For app developers these common set of APIs provided by Spring Data abstracts away the need to learn the database-specific query language, reducing the learning curve required for working with a database.
Spring Data YugabyteDB modules intend to bring support for YugabyteDB YSQL and YCQL API for building modern cloud native applications. For application developers, Spring Data YugabyteDB will bridge the gap for learning the distributed SQL concepts with familiarity and ease of Spring Data APIs.
Goals
YSQL
- Spring Data YugabyteDB Template for YSQL
- Spring Data YugabyteDB repository for YSQL
a. HASH/ASC/DESC options for pkey/index (@id) (non-blocker if not programmatically supported)
b. SPLIT INTO/AT (non-blocker if not programmatically supported)
c. Maybe a uuid/timeuuid/JSON version for @generated key columns
d. Read/write Isolation
e. Read - follower reads
f. Use YugabyteDB JDBC clusteraware driver - Auto reconfiguration support
spring.data.yugabytedb.ysql.*
YCQL
- Spring Data YugabyteDB Template for YCQL
- Spring Data repository support for YCQL and YSQL
a. @enableYugabyteDBYcqlRepositories
b. JSONB data type support for YCQL. Native data type supported in YugabyteDB
c. YugabyteDB YCQL Driver support for PartitionAwarePolicy for One hop fetch
d. Transactional support in YugabyteDB YCQL repository
e. Prepared statement support for YCQL repository - Auto reconfiguration support
spring.data.yugabytedb.ycql.*
Spring Data YugabyteDB
Spring Data YugabyteDB (SDYB) is a top-level project which provides support for both YCQL and YSQL APIs. SDYB will have two projects,
- Spring Data YSQL
- Spring Data YCQL
Spring Data YugabyteDB will have the parent project BOM which will include both these dependencies.
<dependency>
<groupId>com.yugabyte</groupId>
<artifactId>spring-data-yugabytedb-dependencies</artifactId>
<version>1.0.0.RELEASE</version>
</dependency>
Requirements of Spring Data Projects
Spring Data projects should support following database specific configurations,
- Database connectivity
- Externalizing database specific configuration using application.properties
- Database specific data access templates for CRUD operations
- Database specific data access repositories for CRUD operations
- For YCQL -> Implement
@enableyugabyteycqlrepositories
and@enabletransactionsupport
annotation - Support prepared statements in YugabyteDB YCQL repository
- Database specific security implementation
Spring Data YSQL
Spring Data JDBC module provides JDBC template and JDBC repository implementation for relational databases with JDBC access. Spring Data JDBC is not an JPA compliant ORM provider, it's a very lightweight implementation provided by Spring team.
YugabyteDB YSQL Dependency
<dependency>
<groupId>com.yugabyte</groupId>
<artifactId>spring-data-yugabytedb-ysql</artifactId>
<version>1.0.0.RELEASE</version>
</dependency>
Design
-
Implement a YugabyteDB YSQL dialect to support YSQL specific features
a. YB specific annotations for entity mapping and field mapping
- Primary key (Hash/Range)
- Compound keys
- Data types support -
Provide YugabytedbTemplate and YugabytedbRepository implementation
a. Transaction support
b. Follower Reads support
c. Index Support -
Implement
@enableyugabyteysqlrepositories
annotation
Spring Data YCQL
YugabyteDB YCQL API is Cassandra CQL compliant API, YCQL is fully compatible with Spring Data Cassandra project with YugabyteDB YCQL Java driver. However, YugabyteDB has support for additional features which require enhancement to Spring Data projects. Here is the list of enhancements,
- Support for partition awareness, default YugabyteDB session object to use PartitionAwareLoadBalancing Policy
- Support for YugabyteDB ACID Transactions using Spring Transaction Manager
- Support for JSONB data type
- Support for micrometer, prometheus metrics to expose database client metrics
- Auto registering cassandra convertors, UDTs, complex type, JSONB
YugabyteDB YCQL Dependency
<dependency>
<groupId>com.yugabyte</groupId>
<artifactId>spring-data-yugabytedb-ycql</artifactId>
<version>1.0.0.RELEASE</version>
</dependency>
Design
- Create a new Spring Data YCQL project to extend Spring Data cassandra (SDC) project. we can use SDC dependency and update the driver to use YCQL java driver
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-cassandra</artifactId>
<exclusions>
<exclusion>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.yugabyte</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>3.8.0-yb-5</version>
</dependency>
- Externalize configurations properties
- Expose configurations properties as spring.data.yugabytedb.*
Example:
spring.data.yugabytedb.ycql.keyspace-name=cronos
spring.data.yugabytedb.ycql.contact-points=127.0.0.1
spring.data.yugabytedb.ycql.port=9042
- Cross compatibility between Cassandra and YugabyteDB properties
- Mock Testing support for YugabyteDB
-
Provide YugabytedbYCQLTemplate and YugabytedbYCQLRepository implementation
a. Transaction support
b. Index support -
Implement
@enableyugabyteycqlrepositories
and@enabletransactionsupport
annotation
Following tasks, are considered in the first iteration of the Spring Data YugabyteDB project
Spring Data YugabyteDB YSQL Implementation details
Task 1: Adding YugabyteDB Dialect ✅
Task 2: Adding YugabyteDB Specific Configuration ✅
- Make use of YugabyteDB Cluster Aware Datasource ✅
- YugabyteDB specific application properties ✅
- Implement
AbstractYugabytedbYsqlConfiguration
✅ - Application properties:
spring.yugabytedb.ysql.*
⬜️
Task 3: Adding YugabyteDB Template API using YugabyteDB datasource ✅
Task 4: YugabyteDB Specific entity mapping. Extend @id entity annotation to support. [Spring Data JDBC doesn’t support schema generation so not sure if this is required.] ⬜️
- Hash partition
- Range Partition
- Split AT
Option 1: Add properties to existing @id annotation to support above partitioning strategies
Option 2: Add new annotation for Partitioning which works along with @id annotation
@primarykeypartition(value = PartitionType.HASH)
@primarykeypartition(value = PartitionType.RANGE)
Task 5: YugabyteDB datatype support ⬜️
- JSONB
- JSONB index
Task 6: YugabyteDB Repositories ✅
@enableyugabytedbsqlrepositories
annotation ✅- Index hint ⬜️
@follwerread
⬜️- Support for YugabyteDB Distributed Transactions ✅
-@enableyugabytetransactionsupport
- SERIALIZABLE
- SNAPSHOT
Spring Data YugabyteDB Project Structure
spring-data-yugabytedb-parent
→ spring-data-yugabytedb-ysql
→ yugabytedb-ysql-spring-boot-starter
- Current approach is to extend the spring-data-relational and spring-data-jdbc projects. These two projects will be added as a dependency in the
pom.xml
ofspring-data-yugabytedb-ysql
project
Spring Data YugabyteDB - YSQL
Completed Features
- Spring Data YsqlTemplate Support for querying for YugabyteDB
- Spring Data YsqlRepository Support for querying YugabyteDB
- Current iteration of Spring Data YugabyteDB will have support for YSQL
- New @EnableYsqlRepostiory annotation implemented
- Implemented Yugabyte Specific dialect to support YB specific TransactionManager
- Working Spring Boot example using Spring Data YB and YB Smart JDBC Driver
Spring Boot Data YugabyteDB [in-progress]
- Auto reconfiguration support for YB Smart Driver
- Spring starter support for YugabyteDB