This repository contains the Docker images for the following Snowplow components:
They are published in the snowplow-docker-registry.bintray.io
docker registry.
You can pull the images from the registry directly:
# NSQ Scala Stream Collector image, there are others available for Kinesis, Kafka and Google PubSub
docker pull snowplow-docker-registry.bintray.io/snowplow/scala-stream-collector-nsq:0.13.0
# NSQ Stream Enrich image, there are others available for Kinesis, Kafka and Google PubSub
docker pull snowplow-docker-registry.bintray.io/snowplow/stream-enrich-nsq:0.16.1
# Elasticsearch Loader image
docker pull snowplow-docker-registry.bintray.io/snowplow/elasticsearch-loader:0.10.1
# S3 Loader image
docker pull snowplow-docker-registry.bintray.io/snowplow/s3-loader:0.6.0
# Iglu Server image
docker pull snowplow-docker-registry.bintray.io/snowplow/iglu-server:0.3.0
Alternatively, you can build them yourself:
# All images are based on the base image
docker pull snowplow-docker-registry.bintray.io/snowplow/base:0.1.0
# NSQ Scala Stream Collector image, there are others available for Kinesis, Kafka and Google PubSub
docker build -t snowplow/scala-stream-collector-nsq:0.13.0 scala-stream-collector/0.13.0/nsq
# NSQ Stream Enrich image, there are others available for Kinesis, Kafka and Google PubSub
docker build -t snowplow/stream-enrich-nsq:0.16.1 stream-enrich/0.16.1/nsq
# Elasticsearch Loader image
docker build -t snowplow/elasticsearch-loader:0.10.1 elasticsearch-loader/0.10.1
# S3 Loader image
docker build -t snowplow/s3-loader:0.6.0 s3-loader/0.6.0
# Iglu Server image
docker build -t snowplow/iglu-server:0.3.0 iglu-server/0.3.0
Create your own config file filling it according to your setup, you can find examples at the following locations:
- Scala Stream Collector configuration
- Stream Enrich configuration
- Elasticsearch Loader configuration
- S3 Loader configuration
- Iglu Server configuration
Next, you can run a container for each component by mounting your configuration directory:
# NSQ Scala Stream Collector container, there are others available for Kinesis, Kafka and Google PubSub
docker run \
-v $PWD/scala-stream-collector-config:/snowplow/config \
snowplow/scala-stream-collector-nsq:0.13.0 \ # if you have built the image
# snowplow-docker-registry.bintray.io/snowplow/scala-stream-collector-nsq:0.13.0 if you have pulled the image
--config /snowplow/config/config.hocon
# NSQ Stream Enrich container, there are others available for Kinesis, Kafka and Google PubSub
docker run \
-v $PWD/stream-enrich-config:/snowplow/config \
snowplow/stream-enrich-nsq:0.16.1 \ # if you have built the image
# snowplow-docker-registry.bintray.io/snowplow/stream-enrich-nsq:0.16.1 if you have pulled the image
--config /snowplow/config/config.hocon \
--resolver file:/snowplow/config/resolver.json \
--enrichments file:/snowplow/config/enrichments/ \
--force-ip-lookups-download
# Elasticsearch Loader
docker run \
-v $PWD/elasticsearch-loader-config:/snowplow/config \
snowplow/elasticsearch-loader:0.10.1 \ # if you have built the image
# snowplow-docker-registry.bintray.io/snowplow/elasticsearch-loader:0.10.1 if you have pulled the image
--config /snowplow/config/config.hocon
# S3 Loader
docker run \
-v $PWD/s3-loader-config:/snowplow/config \
snowplow/s3-loader:0.6.0 \ # if you have built the image
# snowplow-docker-registry.bintray.io/snowplow/s3-loader:0.6.0 if you have pulled the image
--config /snowplow/config/config.hocon
# Iglu Server
docker run \
-v ${PWD}/iglu-server-config:/snowplow/config \
snowplow/iglu-server:0.3.0 \ # if you have built the image
# snowplow-docker-registry.bintray.io/snowplow/iglu-server:0.3.0 if you have pulled the image
--config /snowplow/config/application.conf
You can find more information in the readme for each image:
- Scala Stream Collector readme
- Stream Enrich readme
- Elasticsearch Loader readme
- S3 Loader readme
- Iglu Server readme
There is a Docker Compose example in the example folder. Iglu Server also has a Docker Compose example in a separate example folder.
Technical Docs | Setup Guide | Roadmap & Contributing |
---|---|---|
Technical Docs | Setup Guide | coming soon |
Copyright 2017-2018 Snowplow Analytics Ltd.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.