Data-Intensive Text Processing
with MapReduce

by Jimmy Lin and Chris Dyer.
Morgan & Claypool Publishers, 2010.

Download book now!

Book Information

Back to top

Table of Contents

  1. Introduction
  2. MapReduce Basics
  3. MapReduce algorithm design
  4. Inverted Indexing for Text Retrieval
  5. Graph Algorithms
  6. EM Algorithms for Text Processing
  7. Closing Remarks

Abstract

Our world is being revolutionized by data-driven methods: access to large amounts of data has generated new insights and opened exciting new opportunities in commerce, science, and computing applications. Processing the enormous quantities of data necessary for these advances requires large clusters, making distributed computing paradigms more crucial than ever. MapReduce is a programming model for expressing distributed computations on massive datasets and an execution framework for large-scale data processing on clusters of commodity servers. The programming model provides an easy-to-understand abstraction for designing scalable algorithms, while the execution framework transparently handles many system-level details, ranging from scheduling to synchronization to fault tolerance. This book focuses on MapReduce algorithm design, with an emphasis on text processing algorithms common in natural language processing, information retrieval, and machine learning. We introduce the notion of MapReduce design patterns, which represent general reusable solutions to commonly occurring problems across a variety of problem domains. This book not only intends to help the reader "think in MapReduce", but also discusses limitations of the programming model as well.

Quite explicitly, this book focuses on MapReduce algorithm design, not Hadoop programming. Tom White's Hadoop: The Definitive Guide is a great resource for learning Hadoop.

Publisher

This book is part of the Morgan & Claypool Synthesis Lectures on Human Language Technologies. If you're at a university, your institution may already subscribe to the series, in which case you can access the electronic version directly without cost (see this page for a list of institutional subscribers). Otherwise, to purchase:

We are pleased to provide the final pre-production manuscript (April 11, 2010) as a preview. If you find this resource helpful, please consider purchasing an actual copy to support our work!

Reference Implementations

Back to top

Cloud9 is a MapReduce library for Hadoop designed to serve as both a teaching tool and to support research in data-intensive text processing. It also serves as a repository of many examples discussed in the book. Reference implementations of design patterns and other algorithms discussed in the book are being added gradually, so please come back periodically. Thus far, the repository contains:

Reviews and Adoption

Back to top

What others are saying...

Courses using this book...