This project implements the stand-alone Data API microservice for Stargate. Data API is an HTTP service that gives access to data stored in a Cassandra cluster using a JSON Document based interface.
Specifications and design documents for this service are defined in the docs directory.
Most users will want to use the Data API through a client library such as the stargate-mongoose for JavaScript development. See the stargate-mongoose-sample-apps for a quick demonstration.
The quickest way to test out the Data API directly is to start a local copy of the service and supporting infrastructure using the Docker compose scripts:
cd docker-compose
./start_hcd.sh
# or
./start_dse69.sh
This starts an instance of the Data API along with a backend database (HCD or DSE 6.9)
Warning Running this script with no options will use the latest
v1
tagged version of Data API. Therefore, if you have these tags already present in your local Docker from other development/testing, those are the images that will be used. See our Docker compose README to see additional options.
Once the services are up, you can access the Swagger endpoint at: http://localhost:8181/swagger-ui/
We also have a Postman collection you can use to learn about the various operations supported by the Data API as part of the Stargate-Cassandra workspace.
There are two main configuration property prefixes used, stargate.
and quarkus.
.
The quarkus.
properties are defined by the Quarkus framework, and the complete list of available properties can be found on the All configuration options page.
In addition, the related guide of each Quarkus extension used in the project provides an overview of the available config options.
The stargate.
properties are defined by this project itself.
The properties are defined by dedicated config classes annotated with the @ConfigMapping
.
The list of currently available properties is documented in the Configuration Guide.
This project uses Quarkus, the Supersonic Subatomic Java Framework. If you want to learn more about Quarkus, please visit its website.
It's recommended that you install Quarkus CLI in order to have a better development experience. See CLI Tooling for more information.
Note that this project uses Java 21, please ensure that you have the target JDK installed on your system.
You can run your application in dev mode that enables live coding using:
docker run -d --rm -e CLUSTER_NAME=dse-cluster -e CLUSTER_VERSION=6.8 -e ENABLE_AUTH=true -e DEVELOPER_MODE=true -e DS_LICENSE=accept -e DSE=true -p 8081:8081 -p 8091:8091 -p 9042:9042 stargateio/coordinator-dse-next:v2
./mvnw compile quarkus:dev -Dstargate.data-store.ignore-bridge=true \
-Dstargate.jsonapi.operations.vectorize-enabled=true \
-Dstargate.jsonapi.operations.database-config.local-datacenter=dc1
The command above will first spin the single Stargate DSE coordinator in dev that the API would communicate to.
NOTE: Quarkus now ships with a Dev UI, which is available in dev mode only at http://localhost:8181/stargate/dev/.
In development mode, Quarkus starts by default with debug mode enabled, listening to port 5005
without suspending the JVM.
You can attach the debugger at any point, the simplest option would be from IntelliJ using Run -> Attach to Process
.
If you wish to debug from the start of the application, start with -Ddebug=client
and create a debug configuration in a Listen to remote JVM mode.
See Debugging for more information.
Integration tests are using the Testcontainers library in order to set up all needed dependencies, a Stargate coordinator and a Cassandra data store.
They are separated from the unit tests and are running as part of the integration-test
and verify
Maven phases:
./mvnw integration-test
Running integration tests from an IDE is supported out of the box. The tests will use the DSE Next as the data store by default. Running a test with a different version of the data store or the Stargate coordinator requires changing the run configuration and specifying the following system properties:
testing.containers.cassandra-image
- version of the Cassandra docker image to use, for example:stargateio/dse-next:4.0.7-336cdd7405ee
testing.containers.stargate-image
- version of the Stargate coordinator docker image to use, for example:stargateio/coordinator-4_0:v2.1
(must be V2.1 coordinator for the target data store)testing.containers.cluster-dse
- optional and only needed if coordinator is used
The integration tests can also be executed against an already running instance of the application.
This can be achieved by setting the quarkus.http.test-host
system property when running the tests.
You'll most likely need to specify the authentication token to use in the tests, by setting the stargate.int-test.auth-token
system property.
./mvnw verify -DskipUnitTests -Dquarkus.http.test-host=1.2.3.4 -Dquarkus.http.test-port=4321 -Dstargate.int-test.auth-token=[AUTH_TOKEN]
You can skip the integration tests during the maven build by disabling the int-tests
profile using the -DskipITs
property:
./mvnw verify -DskipITs
Alternatively you may want to run only integration tests but not unit tests (especially when changing integration tests). This can be achieved using the command:
./mvnw verify -DskipUnitTests
If your Integration Test run fails with some generic, non-descriptive error like:
[ERROR] CollectionsResourceIntegrationTest » Runtime java.lang.reflect.InvocationTargetException
here are some things you should try:
- Make sure your Docker Engine has enough resources. For example following have been observed:
- Docker Desktop defaults of 2 gigabytes of memory on Mac are not enough: try at least 4
The application can be packaged using:
./mvnw package
It produces the quarkus-run.jar
file in the target/quarkus-app/
directory.
Be aware that it’s not an über-jar as the dependencies are copied into the target/quarkus-app/lib/
directory.
The application is now runnable using java -jar target/quarkus-app/quarkus-run.jar
.
If you want to build an über-jar, execute the following command:
./mvnw package -Dquarkus.package.type=uber-jar
The application, packaged as an über-jar, is now runnable using java -jar target/*-runner.jar
.
You can create a Docker image named io.stargate/jsonapi
using:
./mvnw clean package -Dquarkus.container-image.build=true
NOTE: Include the property
-Dquarkus.docker.buildx.platform=linux/amd64,linux/arm64
if you want to build for multiple platforms.
Or, if you want to create a native-runnable Docker image named io.stargate/jsonapi-native
using:
./mvnw clean package -Pnative -Dquarkus.native.container-build=true -Dquarkus.container-image.build=true
You can create a JVM-based Docker image (stargateio/jsonapi-profiling
) with additional profiling tools by using:
./mvnw -B -ntp package -DskipTests -Dquarkus.container-image.build=true -Dquarkus.container-image.name=jsonapi-profiling -Dquarkus.docker.dockerfile-jvm-path=src/main/docker/Dockerfile-profiling.jvm
If you want to learn more about building container images, please consult Container images.
This project uses various Quarkus extensions, modules that run on top of a Quarkus application.
You can list, add and remove the extensions using the quarkus ext
command.
The Quarkus DI solution.
The project uses Docker for building the Container images. Properties for Docker image building are defined in the pom.xml file. Note that under the hood, the generated Dockerfiles under src/main/docker are used when building the images. When updating the Quarkus version, the Dockerfiles must be re-generated.
The extension setups the health endpoints under /stargate/health
.
The OpenAPI definitions are generated and available under /api/json/openapi
endpoint.
The StargateJsonApi class defines the @OpenAPIDefinition
annotation.
This definition defines the default SecurityScheme named Token
, which expects the header based authentication with the HTTP Header Token
.
The info
portions of the Open API definitions are set using the quarkus.smallrye-openapi.info-
configuration properties.
The Data API supports running against a Cassandra backend supporting the latest Storage Attached Index (SAI) features, currently available in DataStax Enterprise (DSE) 6.9 and DataStax Hyper-Converged Database (HCD).
We regularly update the integration tests and Docker compose scripts to support the latest versions of DSE and HCD. To update the version of DSE or HCD that the Data API runs against, the following prodcedure is recommended:
-
Update the
pom.xml
file to reference the correct Docker image. For example, for thedse69-it
profile, update thetesting.containers.cassandra-image
property to the desired version:<stargate.int-test.cassandra.image-tag>6.9.0</stargate.int-test.cassandra.image-tag>
-
Update the
docker-compose
scripts to reference the correct Docker image.-
This includes the
docker-compose/.env
file. For example, you would update theDSETAG
orHCDTAG
as follows:DSETAG=6.9.0
-
Also update the appropriate
docker-compose/start-dse69.sh
ordocker-compose/start-hcd.sh
script to reference the new version as the default, similar to the update in thedocker-compose/.env
file.
-
-
Merge any changes from the reference configuration file in the DSE/HCD repo to the copies in this repo.
- The local copies override the default configuration to set the desired authentication configuration used on the DSE/HCD backend when running in Docker compose or integration tests.
- Sometimes, a new release of DSE/HCD will include new configuration options that we will want to make sure are set in our local configuration, so we don't get out of sync.
- We will want to merge any changes in the source configuration file while preserving our own updates to the authentication configuration.
- For DSE 6.9, you will want to review/update, the files
src/test/resources/dse.yaml
anddocker-compose/dse.yaml
based on the latest version. The two local copies should have the same contents. - For HCD, you will want to review/update, the files
src/test/resources/dse.yaml
anddocker-compose/dse.yaml
based on the latest version. The two local copies should have the same contents.