**PARALLEL COMPUTING:**

Lectures (28h) - topics:

- Lecture 1.
*Introduction: Parallel computers, why parallel computing, application examples, short history, to port or not to port. Performance: overhead, performance metrics for parallel systems* - Lecture 2.
*Performance Metrics for Parallel Programs: analystic modeling, execution time, overhead, speedup, efficiency, cost, granularity, scalability, roadblocks, asymptotic analysis* - Lecture 3.
*Architecture: logical organization - Flynn taxonomy, SIMD, MIMD, communication; physical organization - historical context, shared memory versus distributed memory* - Lecture 4.
*Architecture and Models: physical organization - radius-based classification, multicore, clusters, grids, trends; early models, PRAM* - Lecture 5.
*Models: dataflow and systolic architectures, circuit model, graph model, LogP and LogGP; message-passing paradigm; levels of parallelism* - Lecture 6.
*Implicit Parallelism - Instruction Level Parallelism. Pipeline, Vector and Superscalar Processors* - Lecture 7.
*Cache coherence in multiprocessor systems. Interconnection Networks - classification, topologies, evaluating static and dysnamic interconnection networks* - Lecture 8.
*Communication costs, routing mechanism, mapping techniques, cost-performance tradeoffs* - Lecture 9.
*Concurrency and Steps in Parallel Algoritm Design:concurrency in parallel programs, approaches to achieve concurrency, basic leyers of software concurrency; tasks, processes and processors, design steps,decomposition - simple examples and classification* - Lecture 10.
*Decomposition and Orchestration: recursive, data, exploratoty, speculative and hybrid decompositions, ochestration under the data parallel, shared-address space and message passing model* - Lecture 11.
*Mapping Techniques for Load Balancing and Methods for Containing Interaction Overheads: mapping classification, schemes for static mapping, schemes for dynamic mapping, maximizing data locality, overlapping computations with interactions, replication, optimized collective interactions* - Lecture 12.
*Emulations, Scheduling and Patterns: emulations among architectures, task scheduling problem, scheduling algorithms, load balancing; patterns - task decomposition, data decomposition, group tasks, order tasks, data sharing, design evaluation* - Lecture 13.
*Models of Parallel Algoritms and Simple Parallel Algorithms: models - data parallel, task graph, work pool, master-slave, pipeline, hybrids; applying data parallel model, bilding-block computations; sorting networks* - Lecture 14.
*Parallel computations in numerical analysis: linear equations, nonlinear equations, ordinary differential equations, computational fluid dynamic (slides only in Romanian, book in English).*

Labs (14h) - topics:

- Lab 1
*OpenMP - generalities, simple examples, matrix operations* - Lab 2
*OpenMP - performance studies* - Lab 3
*MPI - generalities and simple examples* - Lab 4
*MPI - matrix operations* - Lab 5
*MPI - performance studies* - Lab 6
*OpenMP - sorting* - Lab 7
*MPI - solving linear systems*

Lectures & Labs - Schedule in Summer Semester of 2016-2017 (subject of changes):

Teaching hours

- Monday 18:00-21:10 In English (Room: 045C)
- Friday 18:00-21:10 in Romanian (Room: 045C)

- Lecture 01: 03.03 Romanian; 27.02 English
- Lecture 02: 10.03 Romanian; 27.02 English
- Lecture 03: 17.03 Romanian; 06.03 English
- Lecture 04: 24.03 Romanian; 06.03 English
- Lecture 05: 31.03 Romanian; 20.03 English
- Lecture 06: 07.04 Romanian; 20.03 English
- Lecture 07: 07.04 Romanian; 27.03 English
- Lecture 08: 28.04 Romanian; 27.03 English
- Lecture 09: 28.04 Romanian; 03.04 English
- Lecture 10: 05.05 Romanian; 03.04 English
- Lecture 11: 12.05 Romanian; 24.04 English
- Lecture 12: 26.05 Romanian; 24.04 English
- Lecture 13: 09.06 Romanian; 08.05 English
- Lecture 14: 09.06 Romanian; 15.05 English

- Lab 1: 03.03 IS+IASTE; 06.03 AIDC; 10.03 IACD
- Lab 2: 17.03 IS+IASTE; 20.03 AIDC; 24.03 IACD
- Lab 3: 31.03 IS+IASTE; 03.04 AIDC; 07.04 IACD
- Lab 4: 25.04 (Tue, 045C, 19:40) IS+IASTE; 24.04 AIDC; 28.04 IACD
- Lab 5: 05.05 IS+IASTE; 08.05 AIDC; 12.05 IACD
- Lab 6: 09.05 (Tue, 045C, 19:40) IS; 23.05 (Tue, 032, 18:00) AIDC; 26:05 IACD+IASTE
- Lab 7: 06.06 (Tue, 045C, 19:40) IS+IASTE; 23.05 (Tue, 045C, 19:40) AIDC; 09.06 IACD

Previous lectures only in Romanian:

- parallel computing lectures for master students
- parallel computer architecture for master students
- parallel computing in numerical analysis for master students
- short version for post-graduate students
- same as above in a printer-friedndly version
- old version for post-graduate students
- short PVM presentation

Textbooks:

- D.Petcu, Parallel computing in English - 2009
- D.Petcu, Parallel processing in Romanian - 2001
- D.Petcu, Parallel algorithms in Romanian - 1994

Links for labs:

- Infrastructure to use: (InfraGrid, GPU Cluster, Blue Gene)
- Connection to InfraGrid and OpenMP
- MPI jobs execution using LoadLeveler

References:

- Bahi Jacques M., Contassot-Vivier Sylvain, Couturier Raphaël eds. Parallel iterative algorithms: from sequential to grid computing, Chapman & Hall/CRC, Taylor & Francis Group, 2008
- Bruaset, Are Magnus, Tveito Aslak Eds.. Numerical Solution of Partial Differential Equations on Parallel Computers, Springer, 2006
- Culler David, Singh Jaswinder Pal, Gupta Anoop. Parallel Computer Architecture. A Hardware/Software Approach, Morgan Kaufmann Publishers, 1997.
- Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin. Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003
- Hughes Cameron, Hughes Tracey. Parallel and Distributed Programming Using C++, Addison Wesley, 2003.
- Karniadakis George E., Kirby Robert M. Parallel Scientific Computing in C++ and MPI, Cambridge University Press, 2003.
- Kontoghiorghes Erricos J. ed. Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, Taylor & Francis Group, 2006
- Lastovetsky Alexey L. Parallel Computing on Heterogeneous Networks, John Wiley & Sons, 2003
- Mattson Timothy G., Sanders Beverly A., Massingill Berna L.Patterns for Parallel Programming, Addison- Wesley Professional, 2004
- Parhami Behrooz. Introduction to Parallel Processing. Algorithms and Architectures, Kluwer Academic Publishers, 2002
- Dana Petcu. Parallel Numerical Algorithms. Part I: Solving systems of linear, nonlinear or differential equations, Mathematical Monographs 60, Printing House of University of Timisoara, 1996.
- Dana Petcu. Parallel Numerical Algorithms. Part II: Solving partial differential equations, Mathematical Monographs 61, TPrinting House of University of Timisoara, 1996.
- Dana Petcu. Parallelism in solving ordinary differential equations, Mathematical Monographs 64, Printing House of University of Timisoara, 1998.
- Petersen W. P., Arbenz P., Introduction to Parallel Computing, Oxford University Press, 2004
- Wittwer Tobias. An Introduction to Parallel Programming, VSSD, Netherlands, 2006

Last modification: February, 2017