58531 - Digital Transmission L-A (6 Credits)

Learning outcomes

Acquisition of the theoretical foundations of Information and Coding Theories. Applications to block codes and convolutional codes. Elements of source coding and applications.

 

Course contents

1) Introduction to Information Theory

Definitions for information and entropy. Joint entropy and conditional entropy. Concatenation rule. Relative entropy. Mutual information. Rules and theorems on entropy and mutual information. Entropy of time discrete random processes. Problems.

 

2) Channel capacity

Capacity of the DMC channel. Properties of capacity. Examples of closed form capacity formulas. Definitions for code, coding rate. Channel coding theorem. Differential entropy for continuous random variables. Entropy of Gaussian variates. Maximum entropy of continuous variates. Time discrete and time continuous AWGN channel capacity. Relationship between spectral efficiency, capacity, and signal-to-noise ratio. Shannon limit. Problems.

 3) Block codes

History of coding. Linear block codes. Systematic codes. Generating matrix. Hamming distance. Properties of block codes. Error detection and correction.

Parity check matrix. Syndrome. Standard array. Code duality, extension, shortening. Examples of block codes. Problems.

 4) Convolutional codes

Definitions and block diagrams. Generating polynomials. State diagram and trellis diagram. Hamming distance, minimum distance, free distance. Algorithm for computing free distance. Code transfer function. Rules for graph reduction. Decoding of convolutional codes. Viterbi algorithm. Performance: union bound. Problems.

 5) Source Coding

Block diagrams. Statistical redundance. Perceptive irrelevance. Classification of source coding methods: lossy and lossless. Variable length coding methods. Prefix constraint. Huffman codes.  Lempel-Ziv algorithm. Problems.

3) Network information theory

Multi-user channels. Broadcast channel capacity. Multiple access channel capacity. MAC/BC duality. Problems.

 

Readings/Bibliography

1. Thomas M. Cover, Joy A. Thomas,
   Elements of Information Theory,
   Wiley Series in Telecommunication 
   
2. John G. Proakis,
   Digital Communications,
   McGRAW-HILL INTERNATIONAL  EDITIONS

Teaching methods

Lectures and classwork. Seminars on advanced topics.

Assessment methods

Written and oral examinations.

Teaching tools

http://gecorazza.deis.unibo.it

Office hours

See the website of Giovanni Emanuele Corazza