Why Shannon’s 1948 Theory Still Powers Modern Communication

The Essence of Communication

Before Shannon, communication engineering was purely empirical; engineers relied on tricks specific to telegraph, telephone, radio, or television. Shannon asked a fundamental question: What is the essence of communication? He answered that communication is the precise (or approximate) reproduction of selected information from one location to another, regardless of the semantic content of the message.

"The fundamental problem of communication is to reproduce, either exactly or approximately, the selected information at one point in another point."

He emphasized that, for engineers, a love letter and a random string of characters are equivalent—they are both information that must be transmitted accurately.

Shannon Communication Model

From this insight, Shannon proposed a simple, universal model of a communication system:

Source → Encoder → Channel → Decoder → Destination
                     ↑
                Noise Source

The model consists of five core components:

Source : Generates the message to be transmitted.

Encoder : Converts the message into a signal suitable for the channel.

Channel : The physical medium that carries the signal.

Decoder : Reconstructs the original message from the received signal.

Destination : The final recipient of the message.

Noise is a key element; real channels are never perfect. Shannon’s genius was to model noise probabilistically and prove that reliable communication is possible even in its presence.

Measuring Information: Bits and Entropy

Shannon asked: How do we measure information? He linked information to uncertainty: learning the outcome of an uncertain event reduces uncertainty, thereby delivering information. The basic unit of information is the bit , representing a binary choice between two equally likely outcomes (e.g., a fair coin toss).

For events with unequal probabilities, Shannon introduced entropy as the average amount of information produced by a source. For a discrete random variable X with probabilities p_i, entropy H(X) is defined as: H(X) = - \sum_i p_i \log_2 p_i Key properties of entropy:

Non‑negativity : Entropy is always ≥ 0, equaling 0 only when one outcome has probability 1.

Maximum at uniform distribution : Entropy is highest when all outcomes are equally likely.

Additivity : The entropy of independent variables adds up.

Higher entropy means greater uncertainty and a higher average information content. For example, English text has an entropy of about 4.1 bits per letter, lower than the 4.7 bits per letter of a completely random sequence, indicating redundancy that can be exploited for compression.

Source Coding Theorem: Limits of Compression

Using entropy, Shannon proved the Source Coding Theorem , which states that any information source can be compressed, but not below its entropy rate. No coding scheme can achieve an average code length shorter than the source entropy, although schemes can approach this limit arbitrarily closely.

This theorem underpins data compression: Morse code, ZIP, JPEG, MP3, and other codecs strive to reach Shannon’s theoretical bound.

Channel Coding Theorem: Reliable Transmission in Noise

Shannon’s most surprising result concerns noisy channels. He showed that every channel has a finite capacity C (bits per second). If the transmission rate R is less than C, there exists a coding scheme that can make the error probability arbitrarily small.

Each channel possesses a "capacity"; as long as the transmission rate stays below this capacity, an appropriate coding scheme can drive the error rate arbitrarily close to zero.

For the additive white Gaussian noise (AWGN) channel, the capacity formula is: C = B \log_2\bigl(1 + \frac{S}{N}\bigr) where B is bandwidth, S is signal power, and N is noise power (the signal‑to‑noise ratio). This relationship guides modern system design: increasing bandwidth or improving SNR raises capacity. Contemporary 5G standards use LDPC and Polar codes that operate within 99 % of Shannon’s limit.

Digitization Is Inevitable

Shannon also concluded that, regardless of the original form—voice, music, image, or video—the most efficient transmission method is to convert the information into bits and send it as a binary stream. This explains why digital communication has become the dominant paradigm across all media.

Seventy‑five years after its publication, Shannon’s “Mathematical Theory of Communication” remains the foundation of information science, influencing everything from early telegraphy to modern quantum communication.