/spline

Data Lineage Tracking And Visualization Solution

Primary LanguageScalaApache License 2.0Apache-2.0

Maven Central TeamCity build (develop) Codacy Badge Sonarcloud Status SonarCloud Maintainability SonarCloud Reliability SonarCloud Security Docker Pulls

Spline — an open-source data lineage tracking solution for data processing frameworks like Apache Spark and others

Watch the video

Documentation

See Spline GitHub Pages

Getting started

See Getting Started

Build project

mvn install

Build Docker containers

See Building Docker

Building from source code

  1. Install Java 11 and Maven 3.6 or above
  2. Run Maven build
# this will produce standard Java artifacts (JAR and WAR files)
mvn install

# or, if you also want Docker images use this command
mvn install -Ddocker -Ddockerfile.repositoryUrl=my

Running Spline server

https://absaoss.github.io/spline/#step-by-step

Versioning strategy

Application version

Spline server follows Semantic Versioning1 principles. The Public API in terms of Semantic Versioning is defined as a combination of API of all Spline modules, including Producer API (REST and Kafka), Consumer REST API, as well as a set of all command-line interfaces (e.g. Admin CLI). Any incompatible change introduced in any of those APIs or CLIs will be accompanied by incrementing the major version component.

Database schema version

The database schema version number does not follow the Semantic Versioning principles, it does not directly correlate with the application version and can only be compared to itself. The only relation between the database schema version number and the application version is that the former indicates in which application version the given database schema was introduced.

How to measure code coverage

./mvn verify -Dcode-coverage

If module contains measurable data the code coverage report will be generated on path:

{local-path}\spline\{module}\target\site\jacoco

Copyright 2019 ABSA Group Limited

you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Footnotes

  1. Semantic Versioning - https://semver.org/ ↩