Syllabus
CS 451/651: Data-Intensive Distributed Computing (Fall 2025)
Schedule
| Week | Description | Dates | Assignments | |
| 1 | The Data Flywheel | Sep 4 | A1 Released: 9/4 | |
| 2 | Data Warehouses, Data Lakes, and Lakehouses | Sep 9, 11 | A2 Released: 9/11 | A1 Due: 9/11 |
| 3 | Batch Processing I | Sep 16, 18 | ||
| 4 | Batch Processing II | Sep 23, 25 | A3 Released: 9/25 | A2 Due: 9/25 |
| 5 | Rubber, Meet Road | Sept 30, Oct 1 | ||
| 6 | Data Infrastructure for Machine Learning | Oct 7, 9 | A4 Released: 10/9 | A3 Due: 10/9 |
| 7 | Reading Week: No Classes! | |||
| 8 | Midterm Exam | Oct 21, 23 | ||
| 9 | Text Processing I | Oct 28, 30 | A5 Released: 10/30 | A4 Due: 10/30 |
| 10 | Text Processing II | Nov 4, 6 | ||
| 11 | Clustering | Nov 11, 13 | A6 Released: 11/13 | A5 Due: 11/13 |
| 12 | Graph Processing | Nov 18, 20 | ||
| 13 | Stream Processing | Nov 25, 27 | A6 Due: 11/27 | |
| 14 | LLMs | Dec 2 | ||
| Final Exam | TBD |
Week 1: The Data Flywheel Sep 4
Key Questions
- What does it mean to be an AI-first or data-driven company?
- What's the data flywheel?
- What's data engineering?
- What are data platforms?
- What are the 4 V's of data?
- What is this course about? And what is it not about?
Readings
- Fundamentals of Data Engineering:
- Designing Data-Intensive Applications, 2nd Edition:
- Chapter 1. Trade-offs in Data Systems Architecture — Intro section and "Analytical versus Operational Systems" (stop at "Cloud versus Self-Hosting")
The above readings are available for free online through the university's library. The links above point directly to Waterloo proxied content, but if you're having trouble accessing the content (e.g., due to VPN settings), you might have go through the library's portal (i.e., search for the book title and follow the appropriate link).
Slides
Week 2: Data Warehouses, Data Lakes, and Lakehouses Sep 9, 11
Key Questions
- What are the main differences between operational and analytical infrastructure?
- What are data warehouses? What problems did they evolve to solve?
- What are data lakes and lakehouses? What problems did they evolve to solve?
- What are the components of modern data platforms?
- How do operational and analytical data models differ?
- What goes on in ETL/ELT?
- How do different physical representations of data affect storage, compute, and other tradeoffs within data platforms?
Readings
- Designing Data-Intensive Applications, 2nd Edition:
- Chapter 3. Data Models and Query Languages — Intro section and "Relational Model versus Document Model" (stop at "Graph-Like Data Models")
- Chapter 4. Storage and Retrieval — Intro section and "Data Storage for Analytics" (i.e., skip "Storage and Indexing for OLTP" and "Multidimensional and Full-Text Indexes")
- Chapter 5. Encoding and Evolution — Intro section and "Formats for Encoding Data" (stop at "Modes of Dataflow")
- (Optional) Fundamentals of Data Engineering (These readings covers similar content as above, but offer a complementary perspective to enrich your perspective):
- Chapter 3. Designing Good Data Architecture — Discussion of data warehouses, ETL, etc.
- Chapter 8. Queries, Modeling, and Transformation — Discussion of data modeling, star and snowflake organizations, etc.
The above readings are available for free online through the university's library. The links above point directly to Waterloo proxied content, but if you're having trouble accessing the content (e.g., due to VPN settings), you might have go through the library's portal (i.e., search for the book title and follow the appropriate link).
- What Goes Around Comes Around... And Around... by Michael Stonebraker and Andrew Pavlo (2024). Answers the question: Why relational?
- What Goes Around Comes Around by Michael Stonebraker and Joseph M. Hellerstein (2005). Optional reading, but provides nice historical historical perspective, written 20 years ago.
- Introduction to Data Lakes from Databricks.
- What is a Data Lake? from Microsoft. (It's helpful to get multiple perspectives, even if some aspects are repetitive)
- What Is a Lakehouse? from Databricks.
- What is a medallion architecture? from Databricks.
Slides
Week 3: Batch Processing I Sep 16, 18
Key Questions
- What's the difference between scaling up and scaling out?
- What are the implications of distributed processing across many machines?
- What are the challenges for a divide-and-conquer strategy?
- What challenges does partitioning address? What challenges does it exacerbate?
- What challenges does replication address? What challenges does it exacerbate?
- What's MapReduce and how does it work with HDFS?
- What challenges do communication and skew present in scaling out?
- Why is local aggregation important?
Readings
- Designing Data-Intensive Applications, 2nd Edition:
- Chapter 2. Defining Nonfunctional Requirements
- Chapter 5. Encoding and Evolution — Intro section and "Formats for Encoding Data" (ties in with 2nd lecture last week)
- Chapter 6. Replication — Intro section to (and including) "Problems with Replication Lag" (Hadoop uses single-leader replication)
- Chapter 7. Sharding — Intro section to (and including) "Request Routing" (Hadoop uses option 2 in Figure 7-7; "Sharding of Key-Value Data" won't be directly relevant until next week, but makes sense to read together)
- Hadoop: The Definitive Guide, 4th Edition:
- Chapter 3. The Hadoop Distributed Filesystem (can safely ignore specific commands, class names, etc.; focus on design)
- Chapter 7. How MapReduce Works (can safely ignore specific commands, class names, etc.; focus on design)
The above readings are available for free online through the university's library. The links above point directly to Waterloo proxied content, but if you're having trouble accessing the content (e.g., due to VPN settings), you might have go through the library's portal (i.e., search for the book title and follow the appropriate link).
The following are optional. They comprise some of the primary sources from which lecture content is drawn, and can enrich your understanding to provide broader context.
- The Datacenter as a Computer: Designing Warehouse-Scale Machines, Third Edition by Luiz André Barroso, Urs Hölzle, Parthasarathy Ranganathan. (This is a long book, but worthwhile to flip through and read anything that catches your attention if you want a better understanding of datacenter design.)
- The original GFS paper
- The original MapReduce paper
Slides
Week 4: Batch Processing II Sep 23, 25
Key Questions
- In what ways does Spark improve over MapReduce?
- How is distributed group by implemented efficiently at scale?
- How do commutative and associative operations contribute to efficient distributed execution?
- How does partitioning contribute to efficient distributed execution?
- How do these concepts come together in efficient joins at scale?
Readings
Reread the assigned readings from last week (or read for the first time if you haven't yet). The material will make a lot more sense given the lecture material. In addition:
- High Performance Spark:
- Data-Intensive Text Processing with MapReduce:
- Chapter 2. MapReduce Basics
- Chapter 3. MapReduce Algorithm Design — the Hadoop-specific material is outdated in the Spark context, but associativity and commutativity remain important.
The above readings are available for free online through the university's library. The links above point directly to Waterloo proxied content, but if you're having trouble accessing the content (e.g., due to VPN settings), you might have go through the library's portal (i.e., search for the book title and follow the appropriate link).
The following are optional. They comprise some of the primary sources from which lecture content is drawn, and can enrich your understanding to provide broader context.
- The original Spark paper
- Paper about Spark SQL
Slides
Week 5: Rubber, Meet Road Sept 30, Oct 1
Readings
The above readings are available for free online through the university's library. The links above point directly to Waterloo proxied content, but if you're having trouble accessing the content (e.g., due to VPN settings), you might have go through the library's portal (i.e., search for the book title and follow the appropriate link).
Slides
Guest Lectures by Khaled Ammar:Week 6: Data Infrastructure for Machine Learning Oct 7, 9
Key Questions
- What are the key components of an ML solution?
- How is the supervised machine learning problem formulated?
- What roles do data platforms and data engineering play?
Readings
- Designing Machine Learning Systems:
- Chapter 2. Introduction to Machine Learning Systems Design — read the section on "Iterative Process", which provides a nice overview of the entire "ML process".
- Chapter 7. Model Deployment and Prediction Service — Intro section, "Machine Learning Deployment Myths", and "Batch Prediction Versus Online Prediction"
- Editorial note: there's a lot of solid practical advice in this book. If you're interested in machine learning, there are many topics of interest; skim/read at your leisure.
- Jimmy Lin and Dmitriy Ryaboy. Scaling Big Data Mining Infrastructure: The Twitter Experience, SIGKDD Explorations, 14(2):6-19, 2012.
The above readings are available for free online through the university's library. The links above point directly to Waterloo proxied content, but if you're having trouble accessing the content (e.g., due to VPN settings), you might have go through the library's portal (i.e., search for the book title and follow the appropriate link).
The following are optional. They provide a deeper dive of lecture content that can enrich your understanding and provide broader context.
- Naive Bayes and Logistic Regression, Tom Mitchell
- Deep Learning: Foundations and Concepts, Christopher M. Bishop and Hugh Bishop [alternative source]
- Chapter 7: Gradient Descent
- Chapter 5.4.3: Logistic Regression