NetLab

 

Semester: II

The classes are organized into two main parts. The first oart is devoted to a general introduction of networked service systems, their modeling and performance evaluation, and optimization of service systems in a deterministic environment (scheduling). The second part of the course focuses on optimization of service systems in a stochastic environment and finally on optimization based on learning from interaction with the system environment (reinforcement learning). Several application examples will be provided throughout the course.

.

The syllabus of the course is as follows.

MODULE 1 (30 hours)

Application context and performance indicators (6): Networked service system modelling. Main general performance indicators. Fundamental trade-offs among utilization efficiency, response time (or, age of information), energy consumption, and accuracy. Examples drawn from telecommunication networks, cloud computing, transportation systems, industrial processes.

Resource management (When the environment is known) (18): Resource sharing: motivations and approaches. Scheduling algorithms and priority handling. Examples of strategic queueing. Scheduling optimization. Congestion and fairness. Network utility maximization: optimization problem statement, distributed solution, game-theoretic perspective.

Applications (6). Examples of optimization applied to telecommunications networks, cloud computing scheduling, transport systems.

MODULE 2 (30 hours)

Stochastic Optimization (When the environment is uncertain) (8): Facing uncertainty: stochastic optimization of networked service systems. Queue-based optimization approach.

Reinforcement Learning (Whne we learn from the environment) (16): Stochastic learning, multi-armed bandit problems, Markov decision processes. Reinforcement Learning: Exploitation vs Exploration, Q-learning, deep reinforcement learning.

Applications (6). Examples of dynamic optimization applied to smart industry, business strategy planning, telecommunication networks, smart grids.

Course Object

1.    Baiocchi, Andrea: Network Traffic Engineering - Stochastic models and applications. Wiley, 2020.

2.    Neely, Michael J. Stochastic network optimization with application to communication and queueing systems. Synthesis Lectures on Communication Networks 3.1 (2010): 1-211.

3.    Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018.

4.    Srikant, Rene, and Lei, Ying. Communication networks – an optimization, control and stochastic networks perspective. Cambridge University Press (2014): Ch. 1,2.

5.    Powell, W.B.: Reinforcement Learning and Stochastic Optimization: A Unified Framework for Sequential Decisions, Princeton

6.    Kelly, F. and Yuodvina, E.: Stochastic Networks. Cambridge University Press, 2014.

7.    Harchol-Balter, M.: Performance modelling and design of computer systems. Cambridge University Press, 2013.

8.    Srikant, R.: The mathematics of Internet congestion control, Birkhauser, 2003.