/snowflake-ingest-java

Java SDK for the Snowflake Ingest Service -

Primary LanguageJavaApache License 2.0Apache-2.0

Snowflake Ingest Service Java SDK

image image image image

The Snowflake Ingest Service SDK allows users to ingest files into their Snowflake data warehouse in a programmatic fashion via key-pair authentication.

Prerequisites

Java 8+

The Snowflake Ingest Service SDK can only be used with Java 8 or higher. Backwards compatibility with Java 7 and prior is not planned at this time.

A 2048-bit RSA key pair

Snowflake Authentication for the Ingest Service requires creating a 2048 bit RSA key pair and, registering the public key with Snowflake. For detailed instructions, please visit the relevant Snowflake Documentation Page.

Maven (Developers only)

This SDK is developed as a Maven project. As a result, you'll need to install maven to build the projects and, run tests.

Adding as a Dependency

You can add the Snowflake Ingest Service SDK by adding the following to your project

<!-- Add this to your Maven project's pom.xml -->
<dependency>
    <groupId>net.snowflake</groupId>
    <artifactId>snowflake-ingest-sdk</artifactId>
    <version>{version}</version>
</dependency>

// in Gradle project
dependencies {
    compile 'net.snowflake:snowflake-ingest-sdk:{version}'
}

Building From Source

If you would like to build this project from source you can run the following to install the artifact to your local maven repository.

mvn install

If you would just like to build the jar in the source directory, you can run

mvn package

However, for general usage, pulling a pre-built jar from maven is recommended.

If you would like to run SnowflakeIngestBasicExample.java in the example folder, you would need to remove the following scope limits in pom.xml

<!-- Remove test scope from snowflake-jdbc -->
<dependency>
    <groupId>net.snowflake<groupId>
    <artifactId>snowflake-jdbc</artifactId>
    <scope>test</scope>
</dependency>

<!-- Remove provided scope from slf4j-api -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <scope>provided</scope>
</dependency>

Testing (SimpleIngestIT Test)

  • Modify TestUtils.java file and replace PROFILE_PATH with profile.json.example for testing.

    • profile.json is used because an encrypted file will be decrypted for Github Actions testing. Check End2EndTest.yml

  • Use an unencrypted version(Only for testing) of private key while generating keys(private and public pair) using OpenSSL.