Appendix B. Installing Storm Cluster

If you want to create a Storm cluster, there are two ways to do so:

  • Use storm-deploy to create a cluster on Amazon EC2, as you saw in Chapter 6.

  • Install Storm manually (more on this in this Appendix).

To install Storm manually, install:

Caution

All of the following steps, the Java, Python and unzip commands will be required in Nimbus and the supervisor process.

When you’ve met the requirements, install the native libraries.

To install ZeroMQ, run:

    wget http://download.zeromq.org/historic/zeromq-2.1.7.tar.gz
    tar -xzf zeromq-2.1.7.tar.gz
    cd zeromq-2.1.7
    ./configure
    make
    sudo make install

To install JZMQ, run:

    git clone https://github.com/nathanmarz/jzmq.git
    cd jzmq
    ./autogen.sh
    ./configure
    make
    sudo make install

Once you have installed the native libraries, download the latest Storm stable version (Storm 0.7.1 at the time of this writing), and unzip it.

Modify the configuration file to add your Storm cluster configuration (you can see all default configurations on defaults.yaml in the Storm repository).

To modify the storm cluster configuration, you’ll need to edit the conf/storm.yaml file located in the Storm folder and set (at least) the following parameters:

storm.zookeeper.servers:
  - "zookeeper addres 1"
  - "zookeeper addres 2"
  - "zookeeper addres N"

storm.local.dir: "a local directory"

nimbus.host: "Numbus host addres"

supervisor.slots.ports:
    - supervisor slot port 1
    - supervisor slot port 2
    - supervisor slot port N

The parameters are the following:

storm.zookeeper.servers

The address of your zookeeper servers.

storm.local.dir

A local directory where the Storm process will store internal data. (It’s very important that the user who runs the storm process has write access over this directory.)

nimbus.host

The address of the machine where the Nimbus will run.

supervisor.slots.ports

The ports (normally starting on 6700) that the workers use for receiving messages; the supervisor will run one worker per port declared into this property.

When you’ve configured these parameters, you can run all the Storm processes. If you want to do a local test, you can configure nimbus.host to localhost.

To start one process, run into the Storm folder: ./bin/storm process name.

Tip

Storm comes with an excellent utility called Storm UI, which enables you to monitor your topologies.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset