Contents

Chapter 1: Big Data and the Hadoop Ecosystem

Big Data Meets Hadoop

The Hadoop Ecosystem

Hadoop Core Components

Hadoop Distributions

Developing Enterprise Applications with Hadoop

Summary

Chapter 2: Storing Data in Hadoop

HDFS

HBase

Combining HDFS and HBase for Effective Data Storage

Using Apache Avro

Managing Metadata with HCatalog

Choosing an Appropriate Hadoop Data Organization for Your Applications

Summary

Chapter 3: Processing Your Data with MapReduce

Getting to Know MapReduce

Your First MapReduce Application

Designing MapReduce Implementations

Summary

Chapter 4: Customizing MapReduce Execution

Controlling MapReduce Execution with InputFormat

Reading Data Your Way with Custom RecordReaders

Organizing Output Data with Custom Output Formats

Writing Data Your Way with Custom RecordWriters

Optimizing Your MapReduce Execution with a Combiner

Controlling Reducer Execution with Partitioners

Using Non-Java Code with Hadoop

Summary

Chapter 5: Building Reliable MapReduce Apps

Unit Testing MapReduce Applications

Local Application Testing with Eclipse

Using Logging for Hadoop Testing

Reporting Metrics with Job Counters

Defensive Programming in MapReduce

Summary

Chapter 6: Automating Data Processing with Oozie

Getting to Know Oozie

Oozie Workflow

Oozie Coordinator

Oozie Bundle

Oozie Parameterization with Expression Language

Oozie Job Execution Model

Accessing Oozie

Oozie SLA

Summary

Chapter 7: Using Oozie

Validating Information about Places Using Probes

Designing Place Validation Based on Probes

Designing Oozie Workflows

Implementing Oozie Workflow Applications

Implementing Workflow Activities

Implementing Oozie Coordinator Applications

Implementing Oozie Bundle Applications

Deploying, Testing, and Executing Oozie Applications

Using the Oozie Console to Get Information about Oozie Applications

Summary

Chapter 8: Advanced Oozie Features

Building Custom Oozie Workflow Actions

Adding Dynamic Execution to Oozie Workflows

Using the Oozie Java API

Using Uber Jars with Oozie Applications

Data Ingestion Conveyer

Summary

Chapter 9: Real-Time Hadoop

Real-Time Applications in the Real World

Using HBase for Implementing Real-Time Applications

Using Specialized Real-Time Hadoop Query Systems

Using Hadoop-Based Event-Processing Systems

Summary

Chapter 10: Hadoop Security

A Brief History: Understanding Hadoop Security Challenges

Authentication

Authorization

Oozie Authentication and Authorization

Network Encryption

Security Enhancements with Project Rhino

Putting it All Together — Best Practices for Securing Hadoop

Summary

Chapter 11: Running Hadoop Applications on AWS

Getting to Know AWS

Options for Running Hadoop on AWS

Understanding the EMR-Hadoop Relationship

Using AWS S3

Automating EMR Job Flow Creation and Job Execution

Orchestrating Job Execution in EMR

Summary

Chapter 12: Building Enterprise Security Solutions for Hadoop Implementations

Security Concerns for Enterprise Applications

What Hadoop Security Doesn’t Natively Provide for Enterprise Applications

Approaches for Securing Enterprise Applications Using Hadoop

Summary

Chapter 13: Hadoop’s Future

Simplifying MapReduce Programming with DSLs

Faster, More Scalable Processing

Security Enhancements

Emerging Trends

Summary

Appendix: Useful Reading

Introduction

Advertisements

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset