1. Handbook
  2. Subjects
  3. Information Theory

Information Theory (ELEN90030)

Graduate courseworkPoints: 12.5Not available in 2024

You’re viewing the 2024 Handbook:
Or view archived Handbooks

Overview

FeesLook up fees

AIMS

Information Theory is the fundamental backbone of reliable communications, reliable data storage, and data compression. This subject provides the rigorous basis of `information', showing it to have deep links to the ability to reduce data to its essence, and to the ultimate limits to communication.

INDICATIVE CONTENT

This subject is aimed at postgraduate (research) students. The subject material covers the core topics of Information Theory including: Shannon entropy, Mutual Information, lossless and lossy source coding, Shannon's celebrated channel capacity and channel coding theorem, differential entropy and the Gaussian channel. Technically the subject combines probabilistic models of idealised communication and the mathematics of applied probability.

Intended learning outcomes

INTENDED LEARNING OUTCOMES (ILOs)


Having completed this subject it is expected that the student be able to:



  • Understand and apply the Shannon notion of entropy to model data coding and communication situations
  • Manipulate Information Theoretic identities and to understand their intuitive meanings
  • Analyse and understand the classic channel models and their capacity analysis in particular discrete memoryless channels, the Gaussian channel, and the critical ideas of random coding and joint typicality
  • Understand the limits to data compression and be able to design codes which can approach the ultimate Shannon limit

Generic skills

On completion of this subject, students will have developed the following skills:

  • Ability to apply knowledge of basic science and engineering fundamentals;
  • In-depth technical competence in at least one engineering discipline;
  • Ability to undertake problem identification, formulation and solution;
  • Ability to utilise a systems approach to design and operational performance;
  • Expectation of the need to undertake lifelong learning, capacity to do so;
  • Capacity for independent critical thought, rational inquiry and self-directed learning;
  • Intellectual curiosity and creativity, including understanding of the philosophical and methodological bases of research activity;
  • Openness to new ideas and unconventional critiques of received wisdom;
  • Profound respect for truth and intellectual integrity, and for the ethics of scholarship.

Last updated: 30 January 2024