Build on Apache Spark DataSourceV2 API.
Goto Home Page to learn how to use this connector.
- Java 8
- Scala 2.12
- Apache Spark 3.2.x
- ClickHouse 21.1.2.15 or newer
Notes:
- Java 11 should work, but not tested.
- Currently, only support gRPC protocol, and ClickHouse support gRPC since v21.1.2.15-stable, but we only do test on v21.8.14.5-lts.
./gradlew clean build -x test
The project leverage Testcontainers and Docker Compose to do integration tests, you should install Docker Desktop and Docker Compose before running test, and check more details on Testcontainers documents if you'd like to run test with remote Docker daemon.
./gradlew clean test
Run single test.
./gradlew test --tests=ConvertDistToLocalWriteSuite