Course detail

Parallel System Architecture and Programming

FIT-ARCAcad. year: 2014/2015

The course covers architecture and programming of parallel systems with functional- and data-parallelism. First the parallel system theory and program parallelization are discussed. Programming for shared memory systems in OpenMP follows and then the most proliferated multi-core multiprocessors (SMP) and the advanced DSM NUMA systems are described.  The course goes on in message passing programming in standardized interface MPI.  Interconnection networks are dealt with separately and then their role in clusters, many-core chips and in the most powerful systems is revealed. In conclusion SIMD accelerators and GPGPU are dealt with. 

Language of instruction

Czech

Number of ECTS credits

5

Mode of study

Not applicable.

Guarantor

prof. Ing. Václav Dvořák, DrSc.

Department

Department of Computer Systems (UPSY)

Learning outcomes of the course unit

Overview of principles of parallel system design and of interconnection networks, communication techniques and algorithms. Survey of parallelization techniques of fundamental scientific problems, knowledge of parallel programming in MPI and OpenMP. The use of SIMD accelerators and GPGPU.

Knowledge of capabilities and limitations of parallel processing, ability to estimate performance of parallel applications. Language means for process/thread communication and synchronization. Competence in hardware-software platforms for high-performance computing and simulations.

Prerequisites

Von-Neumann computer architecture, computer memory hierarchy, cache memories and their organization, programming in assembly and in C/C++.

Co-requisites

Not applicable.

Planned learning activities and teaching methods

Not applicable.

Assesment methods and criteria linked to learning outcomes

To complete successfully term work and be able to write examination, one has to get at least 20 points out of maximum 40.

Course curriculum

    Syllabus of lectures:
    1. Introduction to parallel processing.
    2. Patterns for parallel programming.
    3. Shared memory programming - Introduction into OpenMP.
    4. Synchronization and performance awareness in OpenMP.
    5. Shared memory and cache coherency.
    6. Components of symmetrical multiprocessors.
    7. CC NUMA DSM architectures.
    8. Message passing interface.
    9. Collective communications, communicators, and disk operations.
    10. Interconnection networks: topology and routing algorithms.
    11. Interconnection networks: switching, flow control, message processing and performance.
    12. Message-passing architectures, current supercomputer systems. Distributed file systems.
    13. Data-parallel architectures and programming.

    Syllabus of numerical exercises:
    Tutorials are not scheduled for this course.
    Syllabus - others, projects and individual work of students:
    • Performance prediction of the given parallel application on a compute cluster. 
    • Development of an application on SMP in OpenMP.
    • A parallel program in MPI on the blade cluster.

Work placements

Not applicable.

Aims

To orientate oneself in parallel systems on the market, be able to assess communication and computing possibilities of a particular architecture and to predict the performance of parallel applications. To get acquainted with the most important parallel programming tools (MPI, OpenMP), to learn their practical use and solving problems in parallel.

Specification of controlled education, way of implementation and compensation for absences

Three small projects in duration of 5, 8 a 13 hours ; midterm examination.

Recommended optional programme components

Not applicable.

Prerequisites and corequisites

Not applicable.

Basic literature

Hennessy, J.L., Patterson, D.A.: Computer Architecture - A Quantitative Approach. 5. vydání, Morgan Kaufman Publishers, Inc., 2012, 912 s., ISBN: 9780123747501  Pacecho, P.: Introduction to Parallel Programming. Morgan Kaufman Publishers, 2011, 392 s., ISBN: 9780123742605 

Recommended reading

aktuální PPT prezentace přednášek  http://dolores.sp.cs.cmu.edu/spring2013/ http://www.cs.kent.edu/~jbaker/ParallelProg-Sp11/

Classification of course in study plans

  • Programme IT-MSC-2 Master's

    branch MBI , 0 year of study, summer semester, compulsory-optional
    branch MBS , 0 year of study, summer semester, elective
    branch MIN , 0 year of study, summer semester, elective
    branch MIS , 0 year of study, summer semester, elective
    branch MMI , 0 year of study, summer semester, compulsory-optional
    branch MMM , 0 year of study, summer semester, elective
    branch MPV , 1 year of study, summer semester, compulsory
    branch MSK , 1 year of study, summer semester, compulsory
    branch MGM , 0 year of study, summer semester, compulsory-optional

Type of course unit

 

Lecture

39 hod., optionally

Teacher / Lecturer

prof. Ing. Václav Dvořák, DrSc.

Syllabus

  1. Introduction to parallel processing.
  2. Patterns for parallel programming.
  3. Shared memory programming - Introduction into OpenMP.
  4. Synchronization and performance awareness in OpenMP.
  5. Shared memory and cache coherency.
  6. Components of symmetrical multiprocessors.
  7. CC NUMA DSM architectures.
  8. Message passing interface.
  9. Collective communications, communicators, and disk operations.
  10. Interconnection networks: topology and routing algorithms.
  11. Interconnection networks: switching, flow control, message processing and performance.
  12. Message-passing architectures, current supercomputer systems. Distributed file systems.
  13. Data-parallel architectures and programming.

Project

26 hod., optionally

Teacher / Lecturer

prof. Ing. Václav Dvořák, DrSc.