APPENDIX C
Capacity Tools
Measurement, monitoring, and management tools inform and guide your capacity plan. In this appendix, we’ve compiled a list of some of the more popular tools and utilities for reference.
Monitoring
As we discussed in Chapter 3, there can be a lot of overlap in event notification software (tools that alert on resources based on thresholds) and metric collection and display tools. Some of the following tools have alerting abilities, some of them are more focused on graphing and collection, and some have both.
Metric Collection and Event Notification Systems
-
Born out of the high-performance computing (HPC) community, Ganglia has a very active community of users and developers. We use Ganglia extensively at Flickr, as do Wikipedia and other large-scale social networking sites.
-
GroundWork (GroundWork is a hybrid of Nagios and Ganglia)
Ad Hoc Measurement and Graphing Tools
- RRDTool
- Collectd
-
Scalable system stats collection daemon. Uses multicast, like Ganglia.
- Rrd2csv
-
RRD to CSV converter.
- Dstat
-
System statistics tool, modular.
- GraphClick
-
Digitizer that constructs data from an image of a graph—handy when you have the image but not the raw data.
- Netflix Servo
-
Provides an interface for exposing and publishing application metrics in Java.
Java Performance
- Netflix Vector
-
On-host performance monitoring framework.
Deployment Tools
In this section, we look at tools for automating OS installation, configuration, and cluster management. In addition, we list tools available for inventory management, trend analysis, and curve fitting.
Automated OS Installation
- SystemImager
-
SystemImager comes from the HPC community and is used to install thousand-node computer clusters. Used by many large-scale web operations, as well. Interesting work has been done to use bitTorrent as the transfer mechanism.
- FAI
-
A Debian autoinstallation tool with a healthy community.
- KickStart
- Cobbler
-
Cobbler is a relatively new project from RedHat, supporting RedHat, Fedora, and CentOs.
Configuration Management
- Puppet
-
Fast becoming a very popular configuration tool, Puppet has some very passionate developers and a very involved community of users. Written in Ruby.
- CFEngine
-
Written in C, it’s been around for many years and has a large installed base and active community.
Lcfg (Large-scale Unix configuration system)
Cluster Management/Container Orchestration
- Capistrano
-
Written in Ruby, Capistrano is becoming popular in the Rails environments.
- Func
-
Func is the Fedora Unified Network Controller, and can replace ad hoc cluster-wide ssh commands with an authenticated client/server architecture.
Books on Queuing Theory and the Mathematics of Capacity Planning
-
Leonard Kleinrock. (1975). Queueing Systems. Volume 1: Theory (1st ed.).
-
__ (1976). Queueing Systems. Volume 2: Computer Applications (1st ed.).
-
R. Jain. (1991). The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling.
-
D. A. Menascé and V. A. F. Almeida. (2000). Scaling for E-business. Prentice Hall.
-
__ (2001). Capacity Planning For Web Services: Metrics, Models, and Methods. Prentice Hall.
-
D. A. Menascé et al. (2004). Performance By Design. Prentice Hall.