We may earn an affiliate commission when you visit our partners.
Prof. Raymond W. Yeung
The lectures of this course are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008). This book and its predecessor, A First Course in Information Theory (Kluwer 2002, essentially the first edition of the 2008 book), have been adopted by over 60 universities around the world as either a textbook or reference text.
At the completion of this course, the student should be able to:
1) Demonstrate knowledge and understanding of the fundamentals of information theory.
Read more
The lectures of this course are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008). This book and its predecessor, A First Course in Information Theory (Kluwer 2002, essentially the first edition of the 2008 book), have been adopted by over 60 universities around the world as either a textbook or reference text.
At the completion of this course, the student should be able to:
1) Demonstrate knowledge and understanding of the fundamentals of information theory.
Read more
The lectures of this course are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008). This book and its predecessor, A First Course in Information Theory (Kluwer 2002, essentially the first edition of the 2008 book), have been adopted by over 60 universities around the world as either a textbook or reference text.
At the completion of this course, the student should be able to:
1) Demonstrate knowledge and understanding of the fundamentals of information theory.
2) Appreciate the notion of fundamental limits in communication systems and more generally all systems.
3) Develop deeper understanding of communication systems.
4) Apply the concepts of information theory to various disciplines in information science.
Register for this course and see more details by visiting:
OpenCourser.com/course/sqjjde/information
What's inside
Syllabus
Course Preliminaries
Chapter 1 Information Measures
Chapter 2 Information Measures - Part 1
Read more
Syllabus
Course Preliminaries
Chapter 1 Information Measures
Chapter 2 Information Measures - Part 1
Read more
Chapter 2 Information Measures - Part 2
Chapter 3 The I-Measure - Part 1
Chapter 3 The I-Measure - Part 2
Chapter 4 Zero-Error Data Compression - Part 1
Chapter 4 Zero-Error Data Compression - Part 2
Chapter 5 Weak Typicality
Chapter 6 Strong Typicality
Chapter 7 Discrete Memoryless Channels - Part 1
Chapter 7 Discrete Memoryless Channels - Part 2
Chapter 8 Rate-Distortion Theory - Part 1
Chapter 8 Rate-Distortion Theory - Part 2
Chapter 9 The Blahut-Arimoto Algorithms - Part 1
Chapter 9 The Blahut-Arimoto Algorithms - Part 2
Chapter 10 Differential Entropy - Part 1
Chapter 10 Differential Entropy - Part 2
Chapter 11 Continuous-Valued Channels - Part 1
Chapter 11 Continuous-Valued Channels - Part 2
Chapter 11 Continuous-Valued Channels - Part 3
Good to know
Good to know
Know what's good ,
what to watch for , and
possible dealbreakers
Teaches students about the fundamentals of information theory, a topic highly relevant to communication engineering
Provides a deep understanding of communication systems and their limitations, useful for students pursuing communication engineering
Taught by Prof. Raymond W. Yeung, a renowned expert in information theory
Covers a wide range of topics, from information measures to continuous-valued channels, providing a comprehensive foundation in information theory
Based on Prof. Raymond Yeung’s textbook, a widely recognized resource in the field
Requires strong mathematical background, making it suitable for advanced learners or those with prior knowledge in information theory
Save this course
Save Information Theory to your list so you can find it easily later:
Save
Save
Reviews summary
Information theory: complex but valuable
According to students, Information Theory is a difficult course taught with PowerPoints and readings. Lectures are presented without much, if any, inflection or enthusiasm. One learner remarked that, "all the theorems and concepts were presented in detail, but the assignments make your life miserable."
Course content provides valuable knowledge.
"Very helpful in learning theorems about information theory"
"Excellent course!One downside only: the peer-grading process and the huge percentage of plagiarism. Not more than a few percent (maybe less) of submissions are not plagia. Certificates have no value, but what you learn is quite valuable."
Expect assignments to be difficult.
"D作業死鬼難"
"all the theorems and concepts were presented in detail, but the assignments make your life miserable."
Course material is challenging.
"However, really complicated."
"D作業死鬼難"
Activities
Be better prepared
before
your course. Deepen your understanding
during
and
after
it. Supplement your coursework and achieve mastery of the topics covered
in Information Theory with these
activities:
Review Probability and Random Variables
Show steps
Strengthen your foundation by reviewing probability and random variables, which are essential for understanding information theory.
Browse courses on
Probability
Show steps
-
Review your lecture notes or textbooks on probability and random variables.
-
Complete practice problems on probability distributions, conditional probabilities, and random variables.
Glossary of Information Theory Terms
Show steps
Solidify your understanding of key terms and concepts by creating a comprehensive glossary.
Browse courses on
Information Theory
Show steps
-
Identify key terms and concepts in information theory.
-
Define and explain each term in your own words.
-
Organize the terms into a well-structured glossary.
Information Theory and Network Coding
Show steps
Bolster your understanding of the fundamentals of information theory by reviewing the textbook this course is based on.
Show steps
-
Read the first 11 chapters of the textbook.
-
Take notes on the key concepts covered in each chapter.
-
Complete the practice problems at the end of each chapter.
Five other activities
Expand to see all activities and additional details
Show all eight activities
Weekly Discussion Group
Show steps
Engage with peers to clarify concepts and expand your understanding through discussions.
Browse courses on
Information Theory
Show steps
-
Join a weekly discussion group with other students taking this course.
-
Prepare questions and topics for discussion.
-
Actively participate in discussions and share your insights.
Zero-Error Data Compression Drills
Show steps
Reinforce your understanding of zero-error data compression techniques.
Browse courses on
Data Compression
Show steps
-
Solve practice problems on zero-error data compression algorithms, such as Huffman coding and Lempel-Ziv coding.
-
Implement a simple zero-error data compression algorithm in a programming language of your choice.
Differential Entropy Tutorials
Show steps
Enrich your comprehension of differential entropy, a fundamental concept in information theory.
Show steps
-
Watch video tutorials on differential entropy.
-
Follow online tutorials to derive the properties of differential entropy.
-
Apply differential entropy to calculate the channel capacity of a communication channel.
Rate-Distortion Theory Whitepaper
Show steps
Demonstrate your grasp of rate-distortion theory by creating a succinct whitepaper on its application.
Show steps
-
Research rate-distortion theory and its applications.
-
Choose a specific application and analyze how rate-distortion theory can be used to optimize performance.
-
Write a whitepaper summarizing your research and analysis.
Information Theory Coding Challenge
Show steps
Test your skills and knowledge by participating in a coding challenge that focuses on information theory.
Browse courses on
Information Theory
Show steps
-
Identify an information theory coding challenge.
-
Develop a solution to the challenge.
-
Submit your solution and compete against others.
Review Probability and Random Variables
Show steps
Strengthen your foundation by reviewing probability and random variables, which are essential for understanding information theory.
Browse courses on
Probability
Show steps
Show steps
Show steps
- Review your lecture notes or textbooks on probability and random variables.
- Complete practice problems on probability distributions, conditional probabilities, and random variables.
Glossary of Information Theory Terms
Show steps
Solidify your understanding of key terms and concepts by creating a comprehensive glossary.
Browse courses on
Information Theory
Show steps
Show steps
Show steps
- Identify key terms and concepts in information theory.
- Define and explain each term in your own words.
- Organize the terms into a well-structured glossary.
Information Theory and Network Coding
Show steps
Bolster your understanding of the fundamentals of information theory by reviewing the textbook this course is based on.
Show steps
Show steps
Show steps
- Read the first 11 chapters of the textbook.
- Take notes on the key concepts covered in each chapter.
- Complete the practice problems at the end of each chapter.
Five other activities
Expand to see all activities and additional details
Show all eight activities
Weekly Discussion Group
Show steps
Engage with peers to clarify concepts and expand your understanding through discussions.
Browse courses on
Information Theory
Show steps
Show steps
Show steps
- Join a weekly discussion group with other students taking this course.
- Prepare questions and topics for discussion.
- Actively participate in discussions and share your insights.
Zero-Error Data Compression Drills
Show steps
Reinforce your understanding of zero-error data compression techniques.
Browse courses on
Data Compression
Show steps
Show steps
Show steps
- Solve practice problems on zero-error data compression algorithms, such as Huffman coding and Lempel-Ziv coding.
- Implement a simple zero-error data compression algorithm in a programming language of your choice.
Differential Entropy Tutorials
Show steps
Enrich your comprehension of differential entropy, a fundamental concept in information theory.
Show steps
Show steps
Show steps
- Watch video tutorials on differential entropy.
- Follow online tutorials to derive the properties of differential entropy.
- Apply differential entropy to calculate the channel capacity of a communication channel.
Rate-Distortion Theory Whitepaper
Show steps
Demonstrate your grasp of rate-distortion theory by creating a succinct whitepaper on its application.
Show steps
Show steps
Show steps
- Research rate-distortion theory and its applications.
- Choose a specific application and analyze how rate-distortion theory can be used to optimize performance.
- Write a whitepaper summarizing your research and analysis.
Information Theory Coding Challenge
Show steps
Test your skills and knowledge by participating in a coding challenge that focuses on information theory.
Browse courses on
Information Theory
Show steps
Show steps
Show steps
- Identify an information theory coding challenge.
- Develop a solution to the challenge.
- Submit your solution and compete against others.
Career center
Learners who complete Information Theory will develop knowledge and skills
that may be useful to these careers:
Information Theory Researcher
Information Theory Researcher
Information Theory Researchers design and conduct experiments to develop new theories and technologies in information theory. They use their knowledge of the mathematics of information to improve the efficiency and reliability of communication systems, such as wireless networks and optical fiber systems. Taking courses in information theory can help build a solid foundation in the fundamentals of information theory, including entropy, mutual information, and channel capacity. This knowledge can give you a competitive edge in this field and accelerate your career growth.
Data Scientist
Data Scientist
Data Scientists use their knowledge of statistics, mathematics, and computing to extract insights from large and complex datasets. They use these insights to improve business outcomes, such as increasing sales or reducing costs. Courses in information theory can help Data Scientists develop a stronger understanding of the underlying principles of data analysis, which can lead to better results in their work. For example, they can use the concepts of information theory to identify patterns in data, reduce noise, and improve the accuracy of their models.
Machine Learning Engineer
Machine Learning Engineer
Machine Learning Engineers design, implement, and maintain machine learning models for a variety of applications, such as image recognition, natural language processing, and predictive analytics. A solid understanding of information theory can give you a competitive advantage in this field, as it can help with designing more efficient, robust, and accurate machine learning models. For instance, information-theoretic approaches can be used to select features, improve generalization, and compress models.
Communications Engineer
Communications Engineer
Communications Engineers design, implement, and maintain communication systems, such as wireless networks, optical fiber systems, and satellite systems. A course in information theory can help build a strong foundation in the fundamentals of communication engineering, including modulation, coding, and signal processing. This knowledge can help you develop new and innovative ways to improve the performance of communication systems.
Network Architect
Network Architect
Network Architects design and implement computer networks, including LANs, WANs, and the Internet. A course in information theory can provide a strong foundation in the fundamentals of networking, such as routing, switching, and addressing. This knowledge can help you develop more efficient and reliable networks that can meet the demands of modern applications and services.
Signal Processing Engineer
Signal Processing Engineer
Signal Processing Engineers design and implement algorithms to process and analyze signals, such as audio, video, and data. A course in information theory can help build a strong foundation in the mathematics of signal processing, including Fourier analysis, linear algebra, and probability theory. This knowledge can help you design more efficient and effective signal processing algorithms for a variety of applications.
Information Security Analyst
Information Security Analyst
Information Security Analysts design and implement security measures to protect computer systems and networks from unauthorized access, use, disclosure, disruption, modification, or destruction. A course in information theory can provide a strong foundation in the fundamentals of information security, such as cryptography, authentication, and access control. This knowledge can help you develop more effective security measures to protect systems and networks from cyber attacks.
Computer Scientist
Computer Scientist
Computer Scientists design, develop, and implement computer software and hardware. A course in information theory can provide a strong foundation in the fundamentals of computer science, such as data structures, algorithms, and complexity theory. This knowledge can help you design more efficient and effective computer programs.
Mathematician
Mathematician
Mathematicians develop new mathematical theories and solve problems in a wide range of areas, including algebra, analysis, geometry, and statistics. A course in information theory can provide a strong foundation in the mathematics of information, which can lead to new insights and breakthroughs in other areas of mathematics. For example, information theory has been used to develop better algorithms for solving problems in coding theory and cryptography.
Physicist
Physicist
Physicists study the fundamental laws of nature and how they apply to the physical world. A course in information theory can provide a strong foundation in the mathematics of information, which can be used to study a variety of physical phenomena, such as quantum mechanics and thermodynamics. For example, information theory has been used to develop new theories of quantum information and to improve the efficiency of energy transfer.
Electrical Engineer
Electrical Engineer
Electrical Engineers design and develop electrical systems, such as power systems, control systems, and telecommunications systems. A course in information theory can provide a strong foundation in the mathematics of information, which can be used to design more efficient and reliable electrical systems. For example, information theory has been used to develop new techniques for power system control and to improve the performance of telecommunications networks.
Statistician
Statistician
Statisticians collect, analyze, and interpret data to provide insights for decision making. A course in information theory can provide a strong foundation in the mathematics of information, which can be used to develop more efficient and effective statistical methods. For example, information theory has been used to develop new methods for data compression and to improve the accuracy of statistical models.
Economist
Economist
Economists study the production, distribution, and consumption of goods and services. A course in information theory can provide a strong foundation in the mathematics of information, which can be used to study a variety of economic phenomena, such as market behavior and information asymmetry. For example, information theory has been used to develop new models of market behavior and to design more efficient information markets.
Sociologist
Sociologist
Sociologists study human social behavior and the organization of society. A course in information theory can provide a strong foundation in the mathematics of information, which can be used to study a variety of social phenomena, such as the spread of information and the formation of social networks. For example, information theory has been used to develop new models of social networks and to study the role of information in social change.
Political Scientist
Political Scientist
Political Scientists study the political systems and processes of nations and other political units. A course in information theory can provide a strong foundation in the mathematics of information, which can be used to study a variety of political phenomena, such as the spread of political information and the formation of political coalitions. For example, information theory has been used to develop new models of political campaigns and to study the role of information in political decision-making.
For more career information including salaries, visit:
OpenCourser.com/course/sqjjde/information
Reading list
We've selected 11 books
that we think will supplement your
learning. Use these to
develop background knowledge, enrich your coursework, and gain a
deeper understanding of the topics covered in
Information Theory.
Save
This is the textbook for the course, and it covers all of the topics that will be covered in the course. It great resource for students who want to learn more about information theory.
This is an earlier edition of the textbook for the course. It covers the same material as the current edition, but it is slightly less up-to-date.
Provides a comprehensive overview of information theory, inference, and learning algorithms. It great resource for students who want to learn about the latest advances in these fields.
Provides a comprehensive overview of network information theory. It great resource for students who want to learn about the latest advances in this field.
Provides a comprehensive overview of information theory and statistical learning. It great resource for students who want to learn about the latest advances in these fields.
This classic textbook on information theory. It great resource for students who want to learn more about the mathematical foundations of information theory.
Provides a comprehensive overview of information theory and coding. It great resource for students who want to learn about the mathematical foundations of these fields.
Provides a comprehensive overview of information theory. It great resource for students who want to learn about the mathematical foundations of information theory.
This classic text covers much of the same material as Information Theory and Network Coding. The focus on entropy makes thgreat reference book for topics in data compression and source coding.
Save
Provides a comprehensive overview of information theory and reliable communication. It great resource for students who want to learn about the mathematical foundations of these fields.
Provides a comprehensive overview of coding theory. It great resource for students who want to learn about the mathematical foundations of coding theory.
For more information about how these books relate to this course, visit:
OpenCourser.com/course/sqjjde/information
Share
Similar courses
Here are nine courses similar to
Information Theory.
Introduction to Quantum Information
OpenCourser.com/course/57uqu6/introduction
Introduction to Quantum Information
S210: Corporate Communicati
OpenCourser.com/course/31s997/sayloracademy
S210: Corporate Communicati
BUS300: Operations Management
OpenCourser.com/course/szw3b5/Saylor Academy
BUS300: Operations Management
Computer Security: A Hands-on Approach
OpenCourser.com/course/gsgueu/computer
Computer Security: A Hands-on Approach
Data Science: Probability
OpenCourser.com/course/54e7vf/data
Data Science: Probability
S402: Project Manageme
OpenCourser.com/course/rggrti/sayloracademy
S402: Project Manageme
Crash Course Arduino and Microcontroller Development
OpenCourser.com/course/3eh75n/crash
Crash Course Arduino and Microcontroller Development
Advanced Engineering Systems in Motion: Dynamics of Three Dimensional (3D) Motion
OpenCourser.com/course/7w1msd/advanced
Advanced Engineering Systems in Motion: Dynamics of Three...
MM101: Public Speaki
OpenCourser.com/course/7ubheb/sayloracademy
MM101: Public Speaki
Via
Coursera
Institution
The Chinese University of Hong Kong
Instructor
Prof. Raymond W. Yeung
Language
English
Transcripts
Español
Français
Português
Deutsch
Italiano
中文
العربية
한국어
日本語
हिंदी
Nederlands
Svenska
Pусский
Türkçe
Polski
Ελληνικά
українська мова
Bahasa Indonesia
ไทย
қазақша
Good to know
Teaches students about the fundamentals of information theory, a topic highly relevant to communication engineering
Provides a deep understanding of communication systems and their limitations, useful for students pursuing communication engineering
Taught by Prof. Raymond W. Yeung, a renowned expert in information theory
Covers a wide range of topics, from information measures to continuous-valued channels, providing a comprehensive foundation in information theory
Based on Prof. Raymond Yeung’s textbook, a widely recognized resource in the field
Requires strong mathematical background, making it suitable for advanced learners or those with prior knowledge in information theory
Our mission
OpenCourser helps millions of learners each year. People visit us to learn workspace skills, ace their exams, and nurture their curiosity.
Our extensive catalog contains over 50,000 courses and twice as many books. Browse by search, by topic, or even by career interests. We'll match you to the right resources quickly.
Find this site helpful? Tell a friend about us.
Affiliate disclosure
We're supported by our community of learners. When you purchase or subscribe to courses and programs or purchase books, we may earn a commission from our partners.
Your purchases help us maintain our catalog and keep our servers humming without ads.
Thank you for supporting OpenCourser.
© 2016 - 2024 OpenCourser