[ ![Build Status] travis-image ] travis [ ![Release] release-image ] releases [ license-image ] license
An easily-deployable, single instance version of Snowplow that serves three use cases:
- Gives a Snowplow consumer (e.g. an analyst / data team / marketing team) a way to quickly understand what Snowplow "does" i.e. what you put it at one end and take out of the other
- Gives developers new to Snowplow an easy way to start with Snowplow and understand how the different pieces fit together
- Gives people running Snowplow a quick way to debug tracker updates (because they can)
- Data is tracked and processed in real time
- Added Iglu Server to allow for custom schemas to be uploaded
- Data is validated during processing
- This is done using both our standard Iglu schemas and any custom ones that you have loaded into the Iglu Server
- Data is loaded into Elasticsearch
- Can be queried directly or through a Kibana dashboard
- Good and bad events are in distinct indexes
Snowplow-Mini runs several distinct applications on the same box which are all linked by named pipes. In a production deployment each instance could be an Autoscaling Group and each named pipe would be a distinct Kinesis Stream.
- Scala Stream Collector:
- Starts server listening on port
8080which events can be sent to. - Sends "good" events to the
raw-events-pipe - Sends "bad" events to the
bad-1-pipe
- Starts server listening on port
- Stream Enrich
- Reads events in from the
raw-events-pipe - Sends "good" events to the
enriched-events-pipe - Sends "bad" events to the
bad-1-pipe
- Reads events in from the
- Elasticsearch Sink Good
- Reads events in from the
enriched-events-pipe - Sends the events to the "good" index of the cluster
- On failure to insert writes error to
bad-1-pipe
- Reads events in from the
- Elasticsearch Sink Bad
- Reads events in from the
bad-1-pipe - Sends the events to the "bad" index of the cluster
- Reads events in from the
These events can then be viewed at port 5601 in Kibana.
- Support loading data into Redshift. To give analysts / data teams a good idea to understand what Snowplow "does".
- Create UI to indicate what is happening with each of the different subsystems (collector, enrich etc.), so as to provide developers a very indepth way of understanding how the different Snowplow subsystems work with one another
- [Quick start guide] get-started-guide
Snowplow Mini is copyright 2016 Snowplow Analytics Ltd.
Licensed under the [Apache License, Version 2.0] license (the "License"); you may not use this software except in compliance with the License.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.