Skoltech Course MA06122 (Term 3, 2018-2019)
The aim of the course is to explain basic ideas and results of information and coding theory, some of which has been used for rather long time in data science, in particular various entropy inequalities, and some emerged just very recently, for instance, usage of error-correcting codes for improvements of k-means method for clustering. The course is divided into two parts: introduction to information theory and elements of modern coding theory. In the first part, we consider the measure of information, mutual information, entropy, evaluation of channel capacity for single user and multi-user channels. In the second part, we consider foundations of coding theory such as block codes, linear codes, bounds on the code’s parameters and the most popular algebraic coding methods (Hamming, Reed-Muller, BCH and Reed-Solomon codes). Then we consider modern coding techniques, i.e. iterative decoding systems and graphical models to represent them. Iterative techniques have revolutionized the theory and practice of coding and have been applied in numerous communications standards. We discuss low-density parity-check (LDPC) codes, factor graphs and Sum-Product decoding algorithm.
- Alexey Frolov - Assistant Professor - Link