Back to Teaching

ECE 662: Network Information Theory

Syllabus

This is a second course in information theory, focused on the study of problems involving discrete distributed sources and channels. Our goal in this course is to understand what is known about some classical network problems, and to read some papers and book chapters on interesting related research topics. In this offering, the course will be entirely focused on multiterminal source coding problems.

Part I: Course Introduction and Quick Review of Background Material
Date Lecture on
8/25 - SS Course presentation. Review of ECE 562.
  • Yeung, Chapters 1-4.

  • Cover&Thomas, Chapters 1-5.
Slides for this lecture: [Download]
8/30 - TB Conclude review of ECE 562.
  • Yeung, Chapters 8 and 9.

  • Cover&Thomas, Chapters 8 and 13.
Handouts: [Rate/Distortion] [DMC Capacity - Direct] [DMC Capacity - Converse]

Part II: Point-to-Point Problems with Multiterminal Elements
Date Lecture on
Material Successive Refinements.
  • W. H. R. Equitz, T. M. Cover.
    Successive Refinement of Information.
    IEEE Trans. Inform. Theory, 37(2):269-275, 1991.

  • V. Koshelev.
    Multilevel source coding and data transmission theory.
    Proc. All-Union Conference on Theory of Coding and Data Transmission, Vilnius, USSR, 1978, pt. 1, pp. 85-92.

  • V. Koshelev.
    Hierarchical coding of discrete sources.
    PPI, vol. 16, No. 3, pp. 31-49, 1980.

  • V. Koshelev.
    An evaluation of the average distortion for discrete scheme of sequential compression.
    PPI, vol. 17, No. 3, pp. 20-23, 1981.

  • L. Lastras, T. Berger.
    All Sources are Nearly Successively Refinable.
    IEEE Trans. Inform. Theory, 47(3):918-926, 2001.

  • B. E. Rimoldi.
    Successive Refinement of Information: Characterization of the Achievable Rates.
    IEEE Trans. Inform. Theory, 40(1):253-259, 1994.

  • Yeung, Chapter 5.
Lecture notes: [Download].
Handouts: [Download]
9/01 - SS Successive Refinements.
9/06, 9/08 No lecture (both instructors attend ISIT 2005).
9/13 - SS/TB Successive Refinements.
9/15 - TB Successive Refinements.
9/20 - TB Successive Refinements.
9/22 - TB Successive Refinements.
Materials Multiple Description Coding.
  • A. A. El Gamal, T. M. Cover.
    Achievable Rates for Multiple Descriptions.
    IEEE Trans. Inform. Theory, 28(6):851-857, 1982.

  • More.
Handouts: Parv's completion of a missing step in the proof of the EGC region -- [Download]
9/27 - SS Multiple Description Coding.
9/29 - SS Multiple Description Coding.
10/04 - TB Rate/Distortion with a Remote Source.
  • ...

Part III: Classical Multiterminal Source Coding Problems
Date Lecture on
- General Formulation of Multiterminal Source Coding. Overview of the Main Results.
  • ...

- The Slepian-Wolf Problem.
  • ...

- The Ahlswede-Koerner-Wyner Problem.
  • ...

- The Markov Lemma. Cardinality bounds.
  • ...

10/11 No lecture -- Fall break.
- The Wyner-Ziv Problem.
  • ...

- The Wyner-Ziv Problem.
  • ...

- The Berger-Yeung Problem.
  • ...

- The CEO Problem.
  • ...

- The CEO Problem.
  • ...

- The Berger-Tung Bounds for Multiterminal Source Coding.
  • ...

- Multiterminal Source Coding with Partially Cooperating Encoders.
  • ...

- Convexity Issues.
  • ...

Part IV: To Be Determined.
We have two groups of topics to choose from when covering this part. One option is to revise all the problems above in the context of sources with continuous alphabets, focusing on Gaussian examples. Another option is to use the lectures on the CEO problem as an entry point to the vast literature on functional source coding, estimation based on compressed information, distributed hypothesis testing, etc. (Amari/Berger/Csiszar/Han/Koerner/Marton/etc). We should decide what to do in this regard later in the semester, once we see how things go, if we can stick with this schedule, etc.

About the course

Course staff:

Instructors: Toby Berger, Sergio Servetto.

Meeting times and place, and office hours:

Class: TR 1:25-2:40pm 213 PH.
Office hours: by appointment.

Prerequisites:

Highly recommended: ECE 562, or else at least some probability (e.g., ORIE 651).
Might be of some use: CS 482, MATH 611.

Grading:

75% homework and scribe assignments.
25% presentations.
Extra credit research challenge.

By default, homeworks should be turned in a week after they are assigned, unless either we tell you otherwise or you talk to us and we make arrangements otherwise.

Scribe Instructions:

Jeff Erickson's style file for producing scribe notes.
A sample file using Jeff's style file.

Please note: use Latex only to typeset text. For including figures, just leave a blank space in the file, draw the figures by hand, and scan them. And if you do not have a scanner, just give us a printout and we will take care of scanning them. This will take you far less time to do than using a package like xfig or similar to generate graphics.

Reading materials

Main reference material: Some other useful material: