If you want to create a Storm cluster, there are two ways to do so:
Use storm-deploy to create a cluster on Amazon EC2, as you saw in Chapter 6.
Install Storm manually (more on this in this Appendix).
To install Storm manually, install:
Zookeeper cluster (see how in the Administration Guide)
Java 6.0
Python 2.6.6
Unzip command
All of the following steps, the Java, Python and unzip commands will be required in Nimbus and the supervisor process.
When you’ve met the requirements, install the native libraries.
To install ZeroMQ, run:
wget http://download.zeromq.org/historic/zeromq-2.1.7.tar.gz tar -xzf zeromq-2.1.7.tar.gz cd zeromq-2.1.7 ./configure make sudo make install
To install JZMQ, run:
git clone https://github.com/nathanmarz/jzmq.git cd jzmq ./autogen.sh ./configure make sudo make install
Once you have installed the native libraries, download the latest Storm stable version (Storm 0.7.1 at the time of this writing), and unzip it.
Modify the configuration file to add your Storm cluster configuration (you can see all default configurations on defaults.yaml in the Storm repository).
To modify the storm cluster configuration, you’ll need to edit the conf/storm.yaml file located in the Storm folder and set (at least) the following parameters:
storm.zookeeper.servers: - "zookeeper addres 1" - "zookeeper addres 2" - "zookeeper addres N" storm.local.dir: "a local directory" nimbus.host: "Numbus host addres" supervisor.slots.ports: - supervisor slot port 1 - supervisor slot port 2 - supervisor slot port N
The parameters are the following:
The address of your zookeeper servers.
A local directory where the Storm process will store internal data. (It’s very important that the user who runs the storm process has write access over this directory.)
The address of the machine where the Nimbus will run.
The ports (normally starting on 6700) that the workers use for receiving messages; the supervisor will run one worker per port declared into this property.
When you’ve configured these parameters, you can run all the Storm processes. If you want to do a local test, you can configure nimbus.host to localhost.
To start one process, run into the Storm folder: ./bin/storm process name.
Storm comes with an excellent utility called Storm UI, which enables you to monitor your topologies.