List of Listings

Chapter 2. Core Storm concepts

Listing 2.1. Sample commit data for the GitHub commit feed

Listing 2.2. pom.xml

Listing 2.3. An excerpt from our simple data source: changelog.txt

Listing 2.4. CommitFeedListener.java

Listing 2.5. EmailExtractor.java

Listing 2.6. EmailCounter.java

Listing 2.7. LocalTopologyRunner.java

Chapter 3. Topology design

Listing 3.1. An excerpt from our simple data source, checkins.txt

Listing 3.2. Checkins.java

Listing 3.3. GeocodeLookup.java

Listing 3.4. HeatMapBuilder.java: step 1, collecting incoming tuples into an in-memory map

Listing 3.5. HeatMapBuilder.java: step 2, configuring to receive a signal at a given frequency

Listing 3.6. HeatMapBuilder.java: step 3, emitting the aggregated HeatMap for elapsed time intervals

Listing 3.7. Persistor.java

Listing 3.8. HeatmapTopologyBuilder.java

Listing 3.9. LocalTopologyRunner.java

Listing 3.10. Determining time interval for a tuple in HeatMapBuilder.java

Listing 3.11. TimeIntervalExtractor.java

Listing 3.12. Updating execute() in HeatMapBuilder.java to use the precalculated time interval

Listing 3.13. New bolt added to HeatmapTopologyBuilder.java

Listing 3.14. Adding city as a field in the output tuple of GeocodeLookup.java

Listing 3.15. Pass city field along in TimeIntervalExtractor.java

Listing 3.16. Added second-level grouping to HeatmapTopologyBuilder.java

Chapter 4. Creating robust topologies

Listing 4.1. Order JSON

Listing 4.2. Order.java

Listing 4.3. AuthorizeCreditCard.java

Listing 4.4. ProcessedOrderNotification.java

Listing 4.5. Explicit anchoring and acking in AuthorizeCreditCard.java

Listing 4.6. Anchoring, acking, and failing in AuthorizeCreditCard.execute()

Listing 4.7. ISpout.java interface

Listing 4.8. RabbitMQSpout.java

Listing 4.9. Examining the execute() method of AuthorizeCreditCard.java

Listing 4.10. Updating AuthorizeCreditCard.java to handle a DuplicateTransactionException

Listing 4.11. Updating AuthorizeCreditCard.java to check for database availability before processing

Listing 4.12. Updating AuthorizeCreditCard.java to check the order status before processing

Listing 4.13. Updating AuthorizeCreditCard.java to emit a tuple whenever an order is “processed”

Chapter 5. Moving from local to remote topologies

Listing 5.1. Configuration for our hypothetical Storm cluster

Listing 5.2. CreditCardTopologyBuilder.java for building the credit card authorization topology

Listing 5.3. LocalTopologyRunner.java, which runs the topology on a local cluster

Listing 5.4. RemoteTopologyRunner, which submits the topology to a remote cluster

Chapter 6. Tuning in Storm

Listing 6.1. FlashSaleTopologyBuilder.java

Listing 6.2. CustomerRetrievalSpout.nextTuple generating customer IDs

Listing 6.3. FindRecommendedSales.java

Listing 6.4. LookupSalesDetails.java

Listing 6.5. SaveRecommendedSales.java

Listing 6.6. FindRecommendedSales.java with retry logic

Listing 6.7. LookupSalesDetails.java with metrics

Listing 6.8. SuccessRateMetric.java

Listing 6.9. LookupSalesDetails.java using our new custom metric

Listing 6.10. MultiSuccessRateMetric.java

Listing 6.11. LookupSalesDetails.java with the new MultiSuccessRateMetric

Chapter 7. Resource contention

Listing 7.1. Default settings for supervisor.slots.ports

Listing 7.2. Setting worker.childopts in storm.yaml

Listing 7.3. Configuring the number of worker processes for a topology

Listing 7.4. Setting up GC logging for worker processes

Listing 7.5. Sample GC log output

Listing 7.6. Determining your resource limits

Chapter 8. Storm internals

Listing 8.1. RemoteTopologyRunner.java with configuration for increased buffer sizes

Chapter 9. Trident

Listing 9.1. Building a GitHub commit count Storm topology

Listing 9.2. Building a GitHub commit count Trident topology

Listing 9.3. Sample play log entry for the stream of play logs

Listing 9.4. Wiring up a TransactionalTridentKafkaSpout in the TopologyBuilder

Listing 9.5. TopologyBuilder.java with an each function for deserializing the play logs

Listing 9.6. LogDeserializer.java

Listing 9.7. Splitting the stream originating from LogDeserializer into three separate streams

Listing 9.8. Grouping by artist, title, and tag in the three split streams

Listing 9.9. Built-in Count.java that implements CombinerAggregator.java

Listing 9.10. Using a persistentAggregate operation to update/persist counts in TopologyBuilder.java

Listing 9.11. Creating a DRPC stream

Listing 9.12. Defining the contract for the arguments for the DRPC stream

Listing 9.13. SplitOnDelimiter.java

Listing 9.14. Creating the counts-by-tag stream resulting in TridentState

Listing 9.15. Looking up counts-by-tag by querying a source of state

Listing 9.16. Performing a DRPC request

Listing 9.17. TopologyBuilder.java without the DRPC stream

Listing 9.18. TopologyBuilder.java with named operations

Listing 9.19. The DRPC stream with named operations

Listing 9.20. Specifying parallelism at the points of repartition