Syllabus Big Data Infrastructure (Spring 2015)

Schedule of Classes

Session 1: Introduction January 26

Readings (to be completed before class)

• Data-Intensive Text Processing with MapReduce
  • Chapter 1: Introduction
  • Chapter 2: MapReduce Basics
• Ghemawat et al. (2003) The Google File System. SOSP.
• Dean and Ghemawat. (2004) MapReduce: Simplified Data Processing on Large Clusters. OSDI.
• Barroso et al. (2013) The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines (Second Edition). Morgan & Claypool. (Read at least Chapters 1 and 2, but then read as much of it as you can; at least flip through the remaining chapters.)

Slides

PPTX (Mac)   PDF

Back to top

Session 2: From Business Intelligence to Data Science February 2

Readings (to be completed before class)

• Chaudhuri and Dayal. (1997) An Overview of Data Warehousing and OLAP Technology. ACM SIGMOD Record, 26(1):65-74.
• Hammerbacher. (2009) Information Platforms and the Rise of the Data Scientist. In Beautiful Data: The Stories Behind Elegant Data Solutions. O'Reilly.
• Halevy et al. (2009) The Unreasonable Effectiveness of Data. IEEE Intelligent Systems, 24(2):8-12
• Lin and Ryaboy. (2012) Scaling Big Data Mining Infrastructure: The Twitter Experience. SIGKDD Explorations, 14(2):6-19, 2012.
• Hadoop: The Definitive Guide (3rd Edition):
  • Chapter 1: Meet Hadoop
  • Chapter 2: MapReduce (Skip sections on "Hadoop Streaming" and "Hadoop Pipes".)
  • Chapter 3: The Hadoop Distributed File System (Focus on the mechanics of the HDFS commands and don't worry so much about learning the Java API all at once—you'll pick it up in time. Stop when you get to the section on "Data Ingest with Flume and Sqoop".)

Slides

PPTX (Mac)   PDF

Back to top

Session 3: MapReduce — Basic Algorithm Design February 9

Readings (to be completed before class)

• Data-Intensive Text Processing with MapReduce
  • Chapter 3: MapReduce Algorithm Design
• Ullman. (2012) Designing Good Mapreduce Algorithms. Crossroads.
• Hadoop: The Definitive Guide (3rd Edition):
  • Chapter 4: Hadoop I/O (Read section on "Serialization", pages 93—108 and section on "File-Based Data Structures", pages 130—142)
  • Chapter 5: Developing a MapReduce Application (Skip sections on "Writing a Unit Test with MRUnit" and "Apache Oozie")
  • Chapter 6: How MapReduce Works (Skip section on "Configuration Tuning")
  • Chapter 7: MapReduce Types and Formats
  • Chapter 8: MapReduce Features (Read sections on "Counters", "Side Data distribution", and "Sorting")

Slides

PPTX (Mac)   PDF

Back to top

Session 4: MapReduce — Structured and Unstructured Data February 23

Readings (to be completed before class)

• Data-Intensive Text Processing with MapReduce
  • Chapter 4: Inverted Indexing for Text Retrieval
  • Chapter 6: Processing Relational Data
• Brants et al. (2007) Large Language Models in Machine Translation. EMNLP.
• Elsayed et al. (2010) Brute-Force Approaches to Batch Retrieval: Scalable Indexing with MapReduce, or Why Bother?.

Slides

PPTX (Mac)   PDF

Back to top

Session 5: MapReduce — Graphs March 2

Readings (to be completed before class)

• Data-Intensive Text Processing with MapReduce: Chapter 5: Graph Algorithms
• Cohen. (2009) Graph Twiddling in a MapReduce World. Computing in Science and Engineering.
• Lin and Schatz. (2010) Design Patterns for Efficient Graph Algorithms in MapReduce. MLG.

Slides

PPTX (Mac)   PDF

Back to top

Session 6: MapReduce — Data Mining March 9

Readings (to be completed before class)

• Chapter 3, "Finding Similar Items" of Mining of Massive Datasets by Leskovec, Rajaraman, and Ullman. (Stop reading when you get to 3.4 "Locality-Sensitive Hashing for Documents" on page 87)
• White et al. (2010) Web-Scale Computer Vision using MapReduce for Multimedia Data Mining. MDMKDD.
• Chu et al. (2006) Map-Reduce for Machine Learning on Multicore. NIPS. (This may be a bit dense, but try to get through as much of it as you can.)

Slides

PPTX (Mac)   PDF

Back to top

Session 7: Extending MapReduce March 23

Readings (to be completed before class)

• MapReduce: A major step backwards
• Stonebraker et al. (2010) MapReduce and Parallel DBMSs: Friends or Foes? Communications of the ACM, 53(1):64-71.
• Dean and Ghemawat. (2010) MapReduce: A Flexible Data Processing Tool Communications of the ACM, 53(1):72-77.

Slides

PPTX (Mac)   PDF

Back to top

Session 8: NoSQL March 30

Readings (to be completed before class)

• Chang et al. (2006) Bigtable: A Distributed Storage System for Structured Data. OSDI.
• Khurana (2012) Introduction to HBase Schema Design. Queue.
• Abadi (2012) Consistency Tradeoffs in Modern Distributed Database System Design. IEEE Computer, 45(2):37-42.

Slides

PPTX (Mac)   PDF

Back to top

Session 9: Beyond MapReduce — Dataflow Languages April 6

Readings (to be completed before class)

• Olston et al. (2008) Pig Latin: A Not-So-Foreign Language for Data Processing. SIGMOD.
• Zaharia et al. (2012) Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. NSDI.

Slides

PPTX (Mac)   PDF

Back to top

Session 10: Beyond MapReduce — Graph Processing April 13

Readings (to be completed before class)

• Malewicz et al. (2010) Pregel: A System for Large-Scale Graph Processing. SIGMOD.
• Low et al. (2012) Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud. VLDB.
• Xin et al. (2014) GraphX: Unifying Data-Parallel and Graph-Parallel Analytics. arXiv:1402.2394.

Slides

PPTX (Mac)   PDF

Back to top

Session 11: Beyond MapReduce — Stream Processing April 20

Readings (to be completed before class)

• Zaharia et al. (2013) Discretized Streams: Fault-Tolerant Streaming Computation at Scale. SOSP.
• Boykin et al. (2014) Summingbird: A Framework for Integrating Batch and Online MapReduce Computations. VLDB.
• Toshniwal et al. (2014) Storm @Twitter. SIGMOD.

Slides

PPTX (Mac)   PDF

Back to top