- Docente: Federica Gerace
- Credits: 6
- SSD: MAT/07
- Language: English
- Moduli: Federica Gerace (Modulo 1) Emanuele Mingione (Modulo 2)
- Teaching Mode: Traditional lectures (Modulo 1) Traditional lectures (Modulo 2)
- Campus: Bologna
- Corso: Second cycle degree programme (LM) in Mathematics (cod. 5827)
-
from Sep 16, 2024 to Dec 18, 2024
Learning outcomes
At the end of the course the student : - has in-depth knowledge of the possible applications of complex system's theory to the study of statistical inference and machine learning problems; - is able to introduce a stochastic generative model, set up an inference procedure to extract information from data and discuss process complexity and theoretical limits of the inference/learning performance from the perspective of the theory of complex systems and phase transitions.
Course contents
- Refresh of Probability, Information Theory and Statistical Mechanics;
- Ising Models: thermodynamic states and phase transitions;
- Systems with frustration and Gauge Theory;
- Random Graphs: degree distribution, components and metrics; Configuration Model; Erdos-Renyi; Maximum Entropy Random graphs; Macroscopic Structures and Stochastic Block Model;
- Factor graphs: locally treelike graphs, Bethe Free energy;
- Belief Propagation, Message-Passing Algorithm, TAP equations.
Approfondimenti/Applicazioni
- Ising Spins: Belief Propagation vs Glauber Dynamics;
- Belief Propagation and community detection: detectability transitions;
- Coding, Transmission, Noisy Channels and Decoding;
- Image Restoration;
- Perceptron Learning and Neural Networks: critical capacity and phase transition.
Readings/Bibliography
Main references:
- M.Mézard, A.Montanari - Information, Physics, and Computation - Oxford University Press, USA
(2009); - Nishimori, H.: Statistical Physics of Spin Glasses and Information processing. An Introduction. Oxford
Science Publications 2001; - Coolen, Kuhn, Sollich, Theory of Neural InformationProcessing Systems, Oxford University Press;
- Engel, A., Van Den Broeck, C., Statistical Mechanics of Learning, Cambridge University Press.
Suggested reading:
- Mark Newman - Networks_ An Introduction - Oxford University Press (2010);
- Decelle, A., Krzakala, F., Moore, C., & Zdeborová, L. (2011). Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Physical Review E, 84(6), 066106;
- Zdeborová, L., Krzakala, F. (2016). Statistical physics of inference: Thresholds and
algorithms. Advances in Physics, 65(5), 453-552. - Gardner, E., and Derrida, B.: Optimal storage properties of neural network models. J. Phys. A: Math.
Gen. 21, 271-284 (1988)
Teaching methods
Frontal lectures
Assessment methods
The exam consists of an oral interview in order to verify the knowledge of the arguments listed in the Course Contents and the skills achieved as:
- Advanced Concepts of Applied Statistical Mechanics and Random Graph Theory;
- Ability of reading an optimization, inference, machine learning problem from the statistical mechanics perspective, designing both mathematical structure and possible solutions;
- Ability of performing a numerical experiment, running the studied algorithms on both synthetic and real data;
- Ability of deepening the analyzed topics through the most recent results in the literature.
Teaching tools
The teaching tools will be available on the Virtuale platform
Office hours
See the website of Federica Gerace
See the website of Emanuele Mingione