95601 - Optimization and Machine Learning M

Learning outcomes

The aim of this course is to provide the student the ability of using both machine learning and mathematical optimization for advanced analytics. In particular, machine learning techniques are at the core of predictive analytics, where – based on historical data – one needs to predict the future outcome of a time series or classify unseen observations. Machine learning techniques build upon optimization techniques, and, in the case of neural networks, continuous optimization methods. Conversely, discrete optimization methods are at the core of prescriptive analytics, where one is required to make decisions optimizing a certain revenue / saving function by selecting within a discrete set. Discrete optimization methods are empowered by machine learning in estimating the parameters of the associated decision problems and, in some cases, benefit from machine learning to speed up the solution techniques.

Course contents

 

Students attending this course are supposed to know basic notions on Operations Research, on implementation of algorithms and on the analysis of their computational complexity.

All lectures are given in English. Thus, good comprehension of this language is required.

The course introduces solution algorithms for nonlinear optimization problems that are the basis of many machine learning tools which find applications in telecommunications, electronics, automatic control and decision support systems.

The course is divided into two modules.

Module 1: the first module introduces solution algorithms for nonlinear optimization problems

• Nonlinear optimization: introduction to mathematical programming, models and algorithms.

• Nonlinear models: unconstrained optimization and constrained optimization. Relaxations and penalty algorithms.

• Convex optimization: Lagrangian relaxation and barrier algorithm.

• Applications of convex optimization to support vector machine and deep learning.

Module 2: the second module introduces basic machine learning techniques for classification and regression.

• Introduction to machine learning and data management

• Algorithms for clustering

• Algorithms for classification

• Neural networks

• Interplay between discrete optimization and learning

• Laboratory guidelines to the implementation of learning algorithms

 

Readings/Bibliography

Slides available online

 

For more details:

-- C. Bishop, Pattern Recognition and Machine Learning. Springer

-- S. Boyd e L. Vandenberghe, Convex Optimization. Cambridge University Press
-- L. Grippo e M. Sciandrone, Metodi di Ottimizzazione Non Vincolata. Springer
-- J. Nocedal e S. J. Wright, Numerical Optimization. Springer

Teaching methods

The course consists of class (or online) lectures that concern both the theoretical aspects and the practical application of the algorithms.

Assessment methods

For Module 1, there is a written exam (without books/notes) and/or an oral test (possibly, on the same day).

For Module 2, there is a takeaway project that needs to be completed within one week. The evaluation is based both on a code and a PDF explaining the setting and the results of the projects.

The final grade will be determined according to the grades of both Modules.

 

Teaching tools

All teaching material used during the course will be available to the students on Virtual Learning Environment.

Office hours

See the website of Michele Monaci

See the website of Andrea Lodi