ENG EC 517 - Introduction to Information TheoryPrereq

Institution:

Boston University

Subject:

Description:

ENG EC 381 or equivalent. Discrete memory-less stationary sources and channels; information measures on discrete and continuous alphabets and their properties: entropy, conditional entropy, relative entropy, mutual information, differential entropy; elementary constrained convex optimization; fundamental information inequalities: data-processing, and Fano's; block source coding with outage: weak law of large numbers, entropically typical sequences and typical sets, asymptotic equipartition property; block channel coding with and without cost constraints: jointly typical sequences, channel capacity, random coding, Shannon's channel coding theorem, introduction to practical linear block codes; rate-distortion theory: Shannon's block source coding theorem relative to a fidelity criterion; source and channel coding for Gaussian sources and channels and parallel Gaussian sources and channels (water-filling and reverse water-filling); Shannon's source-channel separation theorem for point-to-point communication; Lossless data compression: Kraft's inequality, Shannon's lossless source coding theorem, variable-length source codes including Huffman, Shannon-Fano-Elias, and arithmetic codes; applications; mini-course project. 4

Credits:

3.00

Credit Hours:

Prerequisites:

Corequisites:

Exclusions:

Level:

Instructional Type:

Lecture

Notes:

Additional Information:

Historical Version(s):

Institution Website:

www.bu.edu

Phone Number:

(617) 353-2000

Regional Accreditation:

New England Association of Schools and Colleges

Calendar System:

Semester

The Course Profile information is provided and updated by third parties including the respective institutions. While the institutions are able to update their information at any time, the information is not independently validated, and no party associated with this website can accept responsibility for its accuracy.