Run: java -cp target/storm-starter-2.4.0.jar -Dstorm.local.sleeptime=20 -Dstorm.jar=target/storm-starter-2.4.0.jar org.apache.storm.LocalCluster org.apache.storm.starter.WordCountTopology
Learn to use Storm!
Table of Contents
First, you need java and git installed and in your user's PATH. Also, two of the examples in storm-starter
require Python and Ruby.
Next, make sure you have the storm-starter code available on your machine. If you have already downloaded storm from http://storm.apache.org/downloads.html then you will find the storm-starter code under your apache-storm-<version>/examples/ directory. Alternatively, Git/GitHub beginners may want to use the
following command to download the latest storm-starter code and change to the new directory that contains the downloaded
code, but make sure you have the same version of storm running.
$ git clone git://github.com/apache/storm.git && cd storm/examples/storm-starter
storm-starter contains a variety of examples of using Storm. If this is your first time working with Storm, check out these topologies first:
- ExclamationTopology: Basic topology written in all Java
- WordCountTopology: Basic topology that makes use of multilang by implementing one bolt in Python
- ReachTopology: Example of complex DRPC on top of Storm
- LambdaTopology: Example of writing spout/bolt using Java8 lambda expression
After you have familiarized yourself with these topologies, take a look at the other topopologies in src/jvm/org/apache/storm/starter/ such as RollingTopWords for more advanced implementations.
If you want to learn more about how Storm works, please head over to the Storm project page.
Install Maven (preferably version 3.x) by following the Maven installation instructions.
If you are using the latest development version of Storm, e.g. by having cloned the Storm git repository,
then you must first perform a local build of Storm itself. Otherwise you will run into Maven errors such as
"Could not resolve dependencies for project org.apache.storm:storm-starter:<storm-version>-SNAPSHOT".
# Must be run from the top-level directory of the Storm code repository
$ mvn clean install -DskipTests=true
This command will build Storm locally and install its jar files to your user's $HOME/.m2/repository/. When you run
the Maven command to build and run storm-starter (see below), Maven will then be able to find the corresponding version
of Storm in this local Maven repository at $HOME/.m2/repository.
You can package a jar suitable for submitting to a Storm cluster with the command:
$ mvn package
This will package your code and all the non-Storm dependencies into a single "uberjar" (or "fat jar") at the path
target/storm-starter-{version}.jar.
Example filename of the uberjar:
>>> target/storm-starter-0.9.3-incubating-SNAPSHOT.jar
You can submit (run) a topology contained in this uberjar to Storm via the storm CLI tool:
# Example 1: Run the ExclamationTopology in local mode (LocalCluster)
$ storm jar target/storm-starter-*.jar org.apache.storm.starter.ExclamationTopology -local
# Example 2: Run the RollingTopWords in remote/cluster mode,
# under the name "production-topology"
$ storm jar target/storm-starter-*.jar org.apache.storm.starter.RollingTopWords production-topology
With submitting you can run topologies which use multilang, for example, WordCountTopology.
You can also submit any topology in local mode via the storm local command, which works much like the storm jar command described above. If you need to run the examples from your IDE, you will need to modify the project slightly, to add a dependency on the storm-server module, and use LocalCluster to create a local cluster you can submit your topology to. Please see the documentation describing Local Mode for a complete overview of this topic.
Use the following Maven command to run the unit tests that ship with storm-starter. Unfortunately lein test does not
yet run the included unit tests.
$ mvn test
The following instructions will import storm-starter as a new project in IntelliJ IDEA.
- Open File > Import Project... and navigate to the storm-starter directory of your storm clone (e.g.
~/git/storm/examples/storm-starter). - Select Import project from external model, select "Maven", and click Next.
- In the following screen, enable the checkbox Import Maven projects automatically. Leave all other values at their defaults. Click Next.
- Make sure to select the intellij profile in the profiles screen. This is important for making sure dependencies set correctly.
- Click Next on the following screen about selecting Maven projects to import.
- Select the JDK to be used by IDEA for storm-starter, then click Next.
- At the time of this writing you should use JDK 7 and above.
- It is strongly recommended to use Oracle JDK rather than OpenJDK.
- You may now optionally change the name of the project in IDEA. The default name suggested by IDEA is "storm-starter". Click Finish once you are done.