Syllabus

Course Objective

CS205 provides an introduction to parallel programming techniques and programming models for applications in science, engineering as well as data science. The class starts with an overview of parallel "machines" based on Flynn's taxonomy and then progresses closer to the computer hardware where various forms of parallelism are identified and discussed through different programming models and techniques. The class focuses on shared memory and distributed memory programming paradigms which are the most useful paradigms when developing software that is targeted for high performance computing (HPC) platforms. CS205 is further concerned about the scaling and performance analysis of parallel code. Both of these analyses are important when writing proposals for compute hours in HPC centers such as the Argonne or Oak Ridge National Labs. The class will discuss methods for performance analysis on the node level and methods for parallel scaling analysis on both node and cluster levels.

A short pdf version of this syllabus can be downloaded here.

Learning Objective

After completing this class, the student is capable of identifying various forms of parallelism in algorithms and exploit them with appropriate techniques. The student will know about methods to analyze the performance and scaling of parallel programs and knows how to further optimize software such that it can be deployed on large scale HPC architectures.

Prerequisites

The course assumes that the student is comfortable with reading and writing code in the C, C++ or Fortran programming languages. Homework, lab and examples in class will be presented using C++. Only simple C++ language features will be utilized for this class. If your proficiency is in C, you should be getting along without trouble. Lecture material for a C/C++ primer class can be found here. Familiarity with Linux command line tools, ssh, git and tools to edit source code is assumed. Prerequisite classes include CS50, CS61, CS107, CS161 or AC207. Either one of these prerequisites or the instructor's permission is required. CS205 will not teach basics of programming.

Textbooks

The class does not follow a specific textbook. The following textbooks are suitable for additional reference:

Course Format

The course contains six main components:

  1. Lectures: Deliver the main content of the class. Attendance is mandatory.
  2. Readings: Accommodate lecture material. The reading assignments are discussed in class. Questions to individual students may be asked.
  3. Quizzes: Graded in-class quizzes intended to assess the learning progress.
  4. Labs: Lab sessions offer practice on topics addressed in class and help support homework assignments. Attendance is mandatory.
  5. Homeworks: Homework assignments deepen the lecture material and include coding exercises (skeleton codes are provided in C++). Exercises may be of theoretical or practical nature, including applications such as simulations based on particle methods or stencil methods on structured grids.
  6. Projects: The class is accompanied by a project (teams of 3-4 students) to practice the methods learned in class on a real application. Topics are proposed by the teams and may involve research problems of individual team members.

Grading

The following weight table is used for individual components of the class. The class does not have standard midterm or final exams.

Total Weight
Homework (5 Homeworks) 40%
Project 35%
Quizzes (4 Quizzes) 10%
Labs (6 Labs for attendance and completion) 10%
Communal Contributions 5%
Bonus (in-class discussions) 5% (max)

Homework

There are 5 homeworks where each contributes equally to the final grade. The homework is focused on the topics discussed in class and involves programming and theoretical work. Programming tasks may be combined with applications such as particle methods or simple grid based finite difference solvers. Familiarity with such methods is not required and explanations provided in the homework statements should be sufficient to solve the tasks. Coding exercises may be implemented in C or C++. Note that some coding exercises will include skeleton codes which are written in C++. It is expected that answers for assignments involving written or theoretical work are typeset in a LaTeX\mathrm{\LaTeX} document. The teaching staff is determined to return solutions and graded assignments with feedback shortly after the due date. It is your responsibility to check the consistency between your graded work and the assignment solution. You have the option to address possible inconsistencies in office hours or request a regrading for the assignment (see the homework grading inconsistencies section below). Homework will be released on the CS205 class repository. Push notifications for that repository will be distributed through the class mailing list.

Homework Submission

Homework must be submitted on Canvas before the due date. Submissions must be a zip or tar archive of the corresponding homework directory (completed with your solutions) which is released through the CS205 class repository. Submissions must only contain source code, text files and written reports in pdf format. Results and data that the problem statement is asking for must be communicated in the written report by means of tables, plots or other forms to display data. Object files and other binary data generated by compilers must be cleaned out before submission. Other binary files that were initially included in the directory can be left in place.

The homework due date is indicated on the problem sheet and displayed in the schedule. Homework submissions will be graded on:

  1. Correctness: your code must compile, run and should produce the correct result. We are not debugging issues when grading submissions.
  2. Presentation: how you present and interpret your results either in a written report or in your code. Presentation means structure and readability. If you present a plot you should have the axes labeled correctly (including units). Difficult sections in code should be commented appropriately such that the teaching staff can understand your thought process. Presentation of results also means that unnecessary or superfluous files like editor backup files or object files left over from previous compilation should not be included in the submission. Results and data must be presented in the written report, the teaching staff will not search for this data in other data files submitted with the homework.

Homework Late Days

Homework submissions are accepted on canvas before the deadline of the assignment is due. You have three late days at your disposal that can be consumed for late submissions and two consecutive late days can be used at most for any of the homework assignments. Deadlines on canvas are enforced and late submissions (after consumption of late days) will not be graded. Submission after the deadline has passed can be done on canvas as well. Please note that you must not update your submission after the deadline has passed as this will render your submission late and will be graded as such. If you must update you correctly submitted solution after the deadline has passed, please contact the teaching staff at cs205-staff@lists.fas.harvard.edu and explain the need to update your solution.

It is your responsibility to plan your work ahead and submit on time. If you have consumed all your late days and you have another late submission, it is in your benefit to still submit the work. We assume the Harvard Honor Code for all late submissions in case solutions are already posted.

If you have a verifiable medical condition or other special circumstances that interfere with your coursework please let us know via cs205-staff@lists.fas.harvard.edu as soon as possible.

Homework Grading Errors

If you believe there is an error in your assignment grading, please submit a regrade request to cs205-staff@lists.fas.harvard.edu.

Note:

  1. The entire assignment will be regraded. This may cause your total grade go up or down.
  2. An assignment can only be regraded once.
  3. Regrade requests are due within 2 days after the release of the grades.

Project

Please see the project section for more details.

Quizzes

There are 4 quizzes in class which are graded and intended to assess the learning progress. Each quiz may address topics from the lecture material in between the previous quiz and the time of the current quiz. Quizzes are open book/www and include multiple choice questions with at most back of the envelope calculations. A quiz will take 15-20 minutes.

Labs

The purpose of the CS205 labs is to provide extended practice on important topics discussed in the lecture which also support homework assignments. Practice is the key to learn and deepen these topics. The labs exists because time constraints do not allow for this extended practice during the lecture. Labs will be offered on different weekdays and lead by the teaching fellows of CS205. You will be asked for your best suitable weekdays according to your schedule such that we can plan the lab offerings accordingly. Your assigned lab day will then be the same throughout the term. There will be 6 labs in total. See the schedule page for more information on lab content and the attendance policy section below.

Lab Submission

Exercises completed in labs must be submitted on Canvas before the due date. Submissions must be a zip or tar archive of the corresponding lab directory (completed with your solutions) which is released through the CS205 class repository. Submissions must only contain source code and text files. Object files and other binary data generated by compilers must be cleaned out before submission. Other binary files that were initially included in the directory can be left in place.

The lab due date is indicated on the problem sheet and displayed in the schedule. Lab submissions will be graded on:

  1. Attendance: your attendance will be recorded by the TF who leads the lab. Joining the lab session at the beginning and then leaving 10-15 minutes later will not reward attendance credit. If you need to leave because of another appointment then it is expected that you communicate before hand and coordinate with your TF. Please see the attendance policy section below as well.
  2. Completion: lab submissions should show effort that the student attempted to solve the tasks. If you experience difficulties in a particular problem and you are not able to complete the task, please indicate the issues you had in your code using comments and we will take that reasoning into account. Just handing in an empty lab skeleton (same as hand-out) does not meet the expected standard and will not award credit for submission.

Communal Contributions

The frequency with which you visit and/or post to the class forum can contribute to your final grade. These contributions must have intellectual value and can be in the form of questions and/or answers related to class material. Be mindful about sharing sensitive material such as solution code for homeworks. If in doubt, try to rephrase your question/answer or ask the teaching staff via cs205-staff@lists.fas.harvard.edu whether the content is permissible to share.

Bonus

You can benefit from a bonus that can contribute to your final grade, see the grading section. This bonus may help you make up for things you either missed or did not perform well otherwise. The bonus grade is determined by optional homework problems.

Reading Assignments

Some lectures have supportive readings assigned (see the schedule). Reading these assignments will help you gain a deeper understanding of the material discussed in class, it will introduce you to new terminology and definitions and it will give you some thoughts and ideas to participate in the class discussion. You are expected to read them but it is not necessary that you thoroughly understand everything. You should typically set aside 1-2 hours for a reading assignment.

Office Hours

The teaching staff holds weekly office hours. Office hour times and locations are listed on the class main page. Office hours provide you with an opportunity to review and discuss course materials as well as provide you with further guidance for your homework.

Attendance Policy

It is expected that when you decide to take CS205 that you also attend the lectures as well as the labs. These are core parts of the class and therefore mandatory to attend. The bonus outlined in the grading section can only be exploited via lecture attendance.

Labs will be held on weekdays that we determine at the beginning of the class according to a best fit of the students' individual schedules for the term. You are required to attend the labs at the assigned lab day. Rescheduling of a lab to a different day due to an unforeseeable event must be coordinated with the teaching staff by sending an email to cs205-staff@lists.fas.harvard.edu.

To be excused from a lecture or a lab, we ask you to follow the Harvard Honor Code and send an email to cs205-staff@lists.fas.harvard.edu at least one day before the lecture or lab. Lecture recordings are available only when students are excused for a lecture.

Zoom

Classes and labs will be streamed via zoom for you to attend real time while in isolation. Note that attendance on zoom is not a substitute for in-person participation. The main attention will be on the physical audience present (unless the entire class must be shifted to virtual meetings depending on Harvard COVID updates). Please follow the steps in the "Attendance Policy" section above to attend the zoom streams. If you are isolating you must not send notifications for every lecture or lab, one notification to let us know is sufficient. Thank you.

Please see the canvas page for the lecture zoom link:

https://canvas.harvard.edu/courses/100601

The zoom links for the lab sessions and office hours are given in the lab_groups.xls and office_hours.xls files in the main class git repository:

https://code.harvard.edu/CS205/main

Collaboration Policy

You are welcome to discuss the course material and homework with others in order to better understand it, but the work you turn in must be your own (with exception of the project where collaborative work is permitted). Any work that is not your own, without properly citing the original author(s), is considered plagiarism. Failure to follow the academic integrity and dishonesty guidelines outlined in the Harvard Student Handbook will have an adverse effect on your final grade. This includes the removal of copyright notices in code. You may not submit the same or similar work to this course that you have submitted or will submit to another without permission. The teaching staff may use tools to compute correlations between submitted work.

Accessibility

Any student receiving accommodations through the Accessible Education Office should present their AEO letter as soon as possible. Failure to do so may prevent us from making appropriate arrangements.

Diversity Statement

All participants in this class are expected to foster empathy and respect towards each other. This includes instructors, teaching staff or students. The motivation to take this course shall be to experience the joy of learning in an environment that allows for a diversity of thoughts, perspectives and experiences and honors your identity including race, gender, class, sexuality, religion, ability, etc. Any constructive feedback for improving the class environment is welcome and I encourage you to reach out to the instructor or teaching staff with any concerns you may have. If you prefer to speak with someone outside of the course, you may find helpful resources at the Harvard Office of Diversity and Inclusion.