Example Storm Topologies
Learn to use Storm!
Table of Contents
Getting started
Prerequisites
First, you need java and git installed and in your user's PATH. Also, two of the examples in storm-starter
require Python and Ruby.
Next, make sure you have the storm-starter code available on your machine. Git/GitHub beginners may want to use the following command to download the latest storm-starter code and change to the new directory that contains the downloaded code.
$ git clone git://github.com/nathanmarz/storm-starter.git && cd storm-starter
storm-starter overview
storm-starter contains a variety of examples of using Storm. If this is your first time working with Storm, check out these topologies first:
- ExclamationTopology: Basic topology written in all Java
- WordCountTopology: Basic topology that makes use of multilang by implementing one bolt in Python
- ReachTopology: Example of complex DRPC on top of Storm
After you have familiarized yourself with these topologies, take a look at the other topopologies in src/jvm/storm/starter/ such as RollingTopWords for more advanced implementations.
If you want to learn more about how Storm works, please head over to the Storm project page.
Using storm-starter with Leiningen
Install Leiningen
The storm-starter build uses Leiningen 2.0. Install Leiningen by following the leiningen installation instructions.
Running topologies with Leiningen
To run a Java topology
$ lein deps
$ lein compile
$ java -cp $(lein classpath) storm.starter.ExclamationTopology
To run a Clojure topology:
$ lein deps
$ lein compile
$ lein run -m storm.starter.clj.word-count
Using storm-starter with Maven
Install Maven
Maven is an alternative to Leiningen. Install Maven (preferably version 3.x) by following the Maven installation instructions.
Running topologies with Maven
storm-starter contains m2-pom.xml which can be used with Maven using the -f option. For example, to
compile and run WordCountTopology in local mode, use the command:
$ mvn -f m2-pom.xml compile exec:java -Dexec.classpathScope=compile -Dexec.mainClass=storm.starter.WordCountTopology
Packaging storm-starter for use on a Storm cluster
You can package a jar suitable for submitting to a Storm cluster with the command:
$ mvn -f m2-pom.xml package
This will package your code and all the non-Storm dependencies into a single "uberjar" at the path
target/storm-starter-{version}-jar-with-dependencies.jar.
Running unit tests
Use the following Maven command to run the unit tests that ship with storm-starter. Unfortunately lein test does not
yet run the included unit tests.
$ mvn -f m2-pom.xml test