Introduction to Distributed and Parallel Programming
|
Instructor
Subodh Kumar <subodh@cse.*> |
Contents: Introduction to concurrency, Race conditions, Atomicity, Semantics of concurrent programs, Examples of distributed algorithms, Client-Server paradigm Parallel architecture, Flynn's classification Shared-memory programming with reference to memory consistency, cache in-coherence, false sharing and mutual exclusion Message passing, High level and collective constructs, Point-to-point communication, multicast and broadcast, Blocking versus non-blocking styles for communication, Message buffering Theoretical models of parallel computation and algorithm analysis, Examples of reduction, prefix-sum Performance metrics: Time, work, Scalability. Task/Communication Dependence graphs, Task decomposition, Data-parallel decomposition, Pipelining Synchronization, barriers, counters/semaphores, Progress, Livelock/Deadlock
| Week | Topic | Reading |
| 01 | Introduction | Ch 4.1, 1.4 |
| 02 | OS & Architecture | Ch 1.1-1.4 |
| 03 | Parallel Architecture and Programming Models | Ch 2 |
| 04 | Memory Consistency and Linearization Relaxed Consistency Models | Ch 4.2 |
| 05 | Concurrency and Synchronization | Ch 4.3 |
| 06,07 |
Distributed Synchronization Message Passing | Ch 6.2 |
| 08 | Performance Measures MPI send semantics | Ch 3.4-3.8 Ch 6.2 |
| 09 | MPI data MPI collectives, One-sided communication | Ch 6.2 |
| 10 | Parallel Fie IO Parallel Pogramming Models |
Ch 2 |
| 11 | Models of Parallel Computation | Ch 3.3 |
| 12 | Manycore Programming | Ch 6.5 |
| 13 | Parallel Algorithms | Ch 7.1-2 |
| 14 | Parallel Algorithms | Ch 7.2-3 |
This means you must protect your code from access by others. Do not leave it where others can find it. Do not give it to someone for submission on your behalf. Do not use any fragment of code obtained online or from someone else, except what is explicitly authorised as a part of the course. Falsifying program output or results is cheating also.
When in doubt, please ask your professor.
Students who are caught cheating will be given a 0 and a full letter-grade
penalty. Second violation will result in summary failure from the course.
| Work | Points | Tentative Schedule |
|---|---|---|
| Assignment 0 | 4 | Due Jan 20, 1159p. |
| Assignment 1 | 5 | Due Feb 12 (17), 1159p. |
| Assignment 2 | 5 | Due Mar 12 (17), 1159p. |
| Assignment 3 | 5 | Due Mar 26 (April 1), 1159p. |
| Assignment 4 | 10 | Due Apr 22 (28), 1159p. |
| Programming Test | 6 | April 23, 5-630pm |
| Quizzes (In class) | 10 | (That includes Mondays) |
| Minor | 25 | |
| Major | 30 |
Late Policy: 0.25 mark/6H of delay, up to 3 days of delay.
Attendance Policy:
Audit Policy: No audit pass.