Example Storm Topologies

Learn to use Storm!

---

Table of Contents

• Getting started
• Using storm-starter with Leiningen
• Using storm-starter with Maven

---

Getting started

Prerequisites

First, you need java and git installed and in your user's PATH. Also, two of the examples in storm-starter require Python and Ruby.

Next, make sure you have the storm-starter code available on your machine. Git/GitHub beginners may want to use the following command to download the latest storm-starter code and change to the new directory that contains the downloaded code.

$ git clone git://github.com/nathanmarz/storm-starter.git && cd storm-starter

storm-starter overview

storm-starter contains a variety of examples of using Storm. If this is your first time working with Storm, check out these topologies first:

1. ExclamationTopology: Basic topology written in all Java
2. WordCountTopology: Basic topology that makes use of multilang by implementing one bolt in Python
3. ReachTopology: Example of complex DRPC on top of Storm

After you have familiarized yourself with these topologies, take a look at the other topopologies in src/jvm/storm/starter/ such as RollingTopWords for more advanced implementations.

If you want to learn more about how Storm works, please head over to the Storm project page.

Using storm-starter with Leiningen

Install Leiningen

The storm-starter build uses Leiningen 2.0. Install Leiningen by following the leiningen installation instructions.

Running topologies with Leiningen

To run a Java topology

$ lein deps
$ lein compile
$ java -cp $(lein classpath) storm.starter.ExclamationTopology

To run a Clojure topology:

$ lein deps
$ lein compile
$ lein run -m storm.starter.clj.word-count

Using storm-starter with Maven

Install Maven

Maven is an alternative to Leiningen. Install Maven (preferably version 3.x) by following the Maven installation instructions.

Running topologies with Maven

storm-starter contains m2-pom.xml which can be used with Maven using the -f option. For example, to compile and run WordCountTopology in local mode, use the command:

$ mvn -f m2-pom.xml compile exec:java -Dexec.classpathScope=compile -Dexec.mainClass=storm.starter.WordCountTopology

Packaging storm-starter for use on a Storm cluster

You can package a jar suitable for submitting to a Storm cluster with the command:

$ mvn -f m2-pom.xml package

This will package your code and all the non-Storm dependencies into a single "uberjar" at the path target/storm-starter-{version}-jar-with-dependencies.jar.

Running unit tests

Use the following Maven command to run the unit tests that ship with storm-starter. Unfortunately lein test does not yet run the included unit tests.

$ mvn -f m2-pom.xml test