This repository collects the materials from the course "Foundations of HPC" at Data Science and Scientific Computer, University of Trieste, year 2020/2021
Teachers:
Stefano Cozzini, Area di Ricerca, Trieste
Luca Tornatore, INAF - National Institute for Astrophysics, Astronomical Observatory of Trieste
Lab session are helped by: Dott. Antonio Ragagnin, INAF - National Institute for Astrophysics, Astronomical Observatory of Trieste
Welcome to the "Foundations of HPC" course at (i) DSSC Department of the University of Trieste and (ii) Master in HPC program at SISSA/ICTP in Trieste.
The scope of the course is to give you:
- an introduction to basic concepts of the High Performance Computing
- an introduction to the modern computer architecture and the techniques to exploit it to maximize performance
- an introduction to parallel/multi-threading programming with MPI and OpenMP
- an introduction to basic and advanced techniques for code optimisation for single- and multi-threaded codes
The lectures will be both in presence (this may change in the future according to the evolution of the COVID pandemic) and in streaming; each lecture will also be recorded and available to be viewed asynchronously. The lectures (the precise calendar is shown in the table below) will be held in ROOMs 3B-H3 and F-A on Wednesday afternoon from 14:00 to 16:00, and Friday morning from 11:00 to 13:00. On Friday afternoon, from 14:00 to 16:00, there will be a lab session with exercises on the topics treated during the lectures.
The lectures will start on Wednesday, 7th October 2020.
| 11.00 - 11.55 | 12.00 - 12.55 | 14.00 - 14.55 | 15.00 - 16.00 | ||
|---|---|---|---|---|---|
| Wednesday | first part | second part | |||
| Friday | first part | second part | lab | lab |
Stefano Cozzini: stefano.cozzini@areasciencepark.it
Luca Tornatore: luca.tornatore@inaf.it
Antonio Ragagnin: antonio.ragagnin@inaf.it
The final grade of the exam will be the result of a written and an oral examination.
During the course, the students have been asked to carry out 3 assignments. It will be required to submit the code developed and the results written in a given format (so to allow automatic checking) on a repo that we will indicate. Each assignment will have a due date of either 1 or 2 weeks.
If the two assignments have been submitted and will be successful, the student will be directly admitted to a final oral examination. The oral examination will focus on the topics covered in the lessons, and it will start from the examination of some of the assignments.
If the either the two assignments will not have been submitted, or they are not successful, a written exam will be required. This test will consist in a project that must be submitted at least 1 week before taking the oral examination.
For submitting 3 successful assignments in the due time before the end of the course, a +1 will be rewarded in the final mark.
| DATE | LECTURES | TUTORIALS | ASSIGMENTS |
|---|---|---|---|
| Wed, Oct 7 | [1] Introduction to course/ introduction to HPC | ||
| Fri, Oct 9 | [2] Introduction to HPC: part 2 | ||
| Wed, Oct 14 | [3] HPC Hardware and parallel computing | ||
| Fri, Oct 16 | [4] HPC Software stack and tools | [1] Using an HPC infrastructure | [1] Assigment on introductory topics (1 week) |
| Wed, Oct 21 | [5] Single Core Optimization (SCO) / 1 Modern Architecture + Opt. Intro + Stack & Heap + Memory allocation |
||
| Fri, Oct 23 | [6] SCO / 2 How to use the Compiler + Cache hierarchy |
[2] Exercises on SCO | |
| Wed, Oct 28 | [7] SCO / 3 loops |
||
| Fri, Oct 30 | [8] SCO / 4 Pipelines +branch predictions |
[3] Tools for profiling | |
| Wed, Nov 4 | [9] SCO / 5 Prefetching + vectorization |
||
| Fri, Nov 6 | [10] SCO / 6 Debugging |
[4] Exercises on SCO | |
| Wed, Nov 11 | [11] Message Passing and intro to MPI programming | ||
| Fri, Nov 13 | [12] Again on MPI programming | [4] Exercises on MPI | |
| Wed, Nov 18 | [13] OpenMP / 1 | ||
| Fri, Nov 20 | [14] OpenMP / 2 | [5] Exercises on OpenMP | |
| Mon, Nov 30 | [15] domain decomposition with MPI | ||
| Mon, Dec 7 | [16] Network Hardware and Performance Model | ||
| Wed, Dec 2 | [17] OpenMP / 3 | ||
| Fri, Dec 4 | [18] OpenMP / 4 | [7] Exercises on OpenMP | |
| Wed, Dec 9 | [19] Performance Model and a short tutorial on measuring MPI performance on ORFEO nodes | ||
| Fri, Dec 11 | [20] OpenMP / 5 | [8] Exercises on MPI and OpenMP | [4] + [5] Assigments on MPI and OpenMP (3 weeks) |
| Fri, Dec 18 | [21] MPI libraries [22] Parallel HPC libraries | [9] Exercises on High Performance Libraries | |
| Fri, Jan 8 | [23] Benchmarking HPC systems | Exercises on benchmarking | |
| Wed, Jan 13 | [24] Parallel Storage | ||
| Fri, Jan 15 | [25] Intro to GPU programming | [10] Exercises on parallel storage | |
| Mon, Jan 18 | [26] roof model + parallel performance analysis | ||
| Fri, Jan 22 | [27] TBD - advanced materials | ||
| Wed, Jan 27 | [28] TBD - advanced materials | ||
| Fri, Jan 29 | [29] TBD - advanced materials |
As you will experiment, having a Linux is always the most adequate solution (Linux is running on the vast majority - possible the 100% - on HPC facilities and brings native tools and utilities for programming).
So, our advice is that on your laptop you run a Linux (of course, it is not needed that you run only Linux: you may a have a multi-boot machine with any other OS you want).
As a second option, you can install a Linux machine in a Virtual box under your preferred OS.
For Windows users, a third option is running Linux under the W10 WSL, that seems to be enough for our course needs.
Available at ICTP library Authors: Thomas Sterling Matthew Anderson Maciej Brodowicz eBook ISBN: 9780124202153 Paperback ISBN: 9780124201583
Some of the materials of this book are presented during some lectures.
The source and pdf of the book (as well as lecture slides) can be found in this repository: https://bitbucket.org/VictorEijkhout/hpc-book-and-course 10.5281/zenodo.49897
Some of the materials presented in classes are taken from this book.
by D. A. Patterson and J. L. Hennessy The Morgan Kaufmann Series in Computer Architecture and Design easily available as pdf on the net.
by Georg Hager and Gerhard Wellein Paperback: 356 pages Publication date July 2, 2010 Editors (Chapman & Hall/CRC Computational Science)
Paperback – October 15, 2014 by Alexander Supalov (Author), Andrey Semin (Author), Michael Klemm (Author), & 1 more ISBN-13: 978-1430264965 ISBN-10: 1430264969 Edition: 1st
Multicore and Many-core Programming Approaches by Reinders & Jeffers Morgan Kaufmann, Print Book ISBN :9780128021187