Staff

The instructor for this course is Jimmy Lin. If you're interested in my research, here is my homepage. The course TAs are Nafis Ahmed and Eimaan Saqib.

If you have general questions, post on Piazza. If you have personal concerns, send us a private post on Piazza (private posts are only visible by me and the TAs). Unless you specifically have a reason to email me directly, please use Piazza.

Course Materials

The most recent version of all materials for this course will be posted on this website, including the syllabus, readings, slides, and assignments.

‼️ You are responsible for keeping up to date with all materials posted on this website — check frequently for updates. While we will strive to announce important changes, ultimately it is your responsibility. "I didn't see it" or "I didn't know" will not be accepted as an excuse.

Readings for this course will be assigned from a number of sources, including academic papers and the following books:

• Designing Data-Intensive Applications, 2nd Edition, by Martin Kleppmann and Chris Riccomini. (2026) Note that the 2nd edition is in "early release"; it is not officially published yet but content is already available.
• Fundamentals of Data Engineering, by Joe Reis, Matt Housley. (2022)

Both of these books are available for free online through the university's library. The links above point directly to Waterloo proxied content, but if you're having trouble accessing the content (e.g., due to VPN settings), you might have go through the library's portal (i.e., search for the book title and follow the appropriate link).

For learning Spark, sources abound on the web, but if you prefer books, O'Reilly Media provides the following, also available for free through the university's library:

• Learning Spark, 2nd Edition, by Jules S. Damji, Brooke Wenig, Tathagata Das, Denny Lee. (2020)

Note the publication date of the book and the software versions it covers: be aware that APIs, methods, invocations, etc. change from version to version.

Grading

Components of the final grade for CS 451 are as follows:

Components of the final grade for CS 651 are as follows:

The homework assignments are to be completed individually. See below section on Academic Integrity. Assignments deadlines are clearly stated on the syllabus.

Late policy: For assignments up to 24 hours late, we will take the grade you would have gotten and multiply it by 0.75 (i.e., 25% reduction). For assignments more than 24 hours late but less than 48 hours late, we will take the grade you have gotten and multiply it by 0.5 (i.e., 50% reduction). Assignments more than 48 hours late will not be accepted. In all cases, we will round down. By assumption, we will mark your assignment at the deadline   if you want us to "hold off" on marking (and take the late penalty), you must let us know, and you must let us know when the assignment is ready for marking so we can compute the late penalty appropriately. For these communications, use private posts on Piazza.

Assignment marking reappraisal requests: If you believe we have made an error marking your assignment, you may request that your assignment be reappraised. Please send a private post on Piazza with subject "Remark Request: A?-userid", for example, "Remark Request: A4-n85ahmed". In your post, clearly describe the reason for the remark request. Note that for each request, the entire assignment will be reconsidered, in addition to the highlighted issues. This means that your grade might be adjusted up or down — the latter case if we found an error that was missed the first time.

Late enrollment policy: Students who enroll late into the course must follow the lectures, complete the readings, and hand in all assignments on time. That is, (barring extraordinary circumstances) late enrollment is not an acceptable excuse for late assignments, and will be subjected to the same late penalties as above.

Academic Integrity

All work in this course is to be done individually unless otherwise noted.

‼️ You are permitted to use generative AI (ChatGPT, etc.) for your assignments in the same way you would use a search engine. That is, you can query for answers to specific questions, look up APIs, etc. However, it is not okay to just copy/paste the entire assignment (or significant portions thereof, or entire questions, etc.) into the prompt and ask the AI to do your assignment. If you have any questions as to what is acceptable, ask. For every assignment, you must tell us exactly how you used generative AI (as part of the assignment submission). A failure to disclose use will be treated as unauthorized collaboration, even if the original use falls within the bounds of acceptable uses specified here.

Incidents of suspected plagiarism or unauthorized collaboration will be reported to the office of the Associate Dean for Undergraduate Studies. The standard penalty for a first offence is a grade of zero for the assessment in question and a 5-mark deduction from the course grade. Note that in cases in which one individual provides assistance to another, the same penalty is normally applied to each. Subsequent offences are likely to result in more severe penalties, possibly including suspension or even expulsion.

To avoid inadvertently plagiarizing, you should discuss assignment issues with other students only in a very broad and high-level fashion. Do not take notes during such discussions and do not look at anyone else's assignment code, on screen or on paper. If you find yourself stuck, contact the TAs or the instructor for help. You are allowed to search the web for information about general issues and use generative AI in the manner described above. If you inadvertently stumble onto a solution to any of the assignments online, however, please let us know — we will appreciate it and not construe it as plagiarism (unless, of course, you actually do copy the solution).

Senate Undergraduate Council has asked us to post the following paragraphs:

Academic Integrity: In order to maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility.

Grievance: A student who believes that a decision affecting some aspect of his/her university life has been unfair or unreasonable may have grounds for initiating a grievance. Read Policy 70 - Student Petitions and Grievances, Section 4.

Discipline: A student is expected to know what constitutes academic integrity, to avoid committing academic offenses, and to take responsibility for his/her actions. A student who is unsure whether an action constitutes an offense, or who needs help in learning how to avoid offenses (e.g., plagiarism, cheating) or about "rules" for group work/collaboration should seek guidance from the course professor, academic advisor, or the Undergraduate Associate Dean. When misconduct has been found to have occurred, disciplinary penalties will be imposed under Policy 71 - Student Discipline. For information on categories of offenses and types of penalties, students should refer to that policy.

Avoiding Academic Offenses: Most students are unaware of the line between acceptable and unacceptable academic behaviour, especially when discussing assignments with classmates and using the work of other students. For information on commonly misunderstood academic offenses and how to avoid them, students should refer to the Faculty of Mathematics Cheating and Student Academic Discipline Policy.

Appeals: A student may appeal the finding and/or penalty in a decision made under Policy 70 - Student Petitions and Grievances (other than regarding a petition) or Policy 71 - Student Discipline if a ground for an appeal can be established. Read Policy 72 - Student Appeals.

Accommodations

Illness policy: From time to time students become ill or have ongoing medical conditions that prevent them from meeting academic obligations. For these cases, please consult the university policy.

Accommodations for Religious Holidays and Other Special Circumstances. Students wishing to discuss accommodations for religious holidays on dates that assignments are due, or other circumstances not addressed in this course information page, should discuss those circumstances with me before the third class session in order to permit adequate time for planning. Only accommodations for unforeseeable circumstances will be considered after that date.

Note for students with disabilities: AccessAbility Services collaborates with all academic departments to arrange appropriate accommodations for students with disabilities without compromising the academic integrity of the curriculum. If you require academic accommodations to lessen the impact of your disability, please register with them at the beginning of each academic term.