Master KL Divergence: Definitions, Properties, and Real‑World Applications

Definition

For discrete distributions P and Q, the Kullback‑Leibler (KL) divergence is defined as

For continuous distributions the definition is

D_KL(P||Q)

measures the amount of information (in bits) lost when Q approximates P. It equals zero only when the two distributions are identical.

D_KL(P||Q)=0 only if P = Q

In information theory, KL divergence equals the difference between entropy H(p) and cross‑entropy H_ce(p,q), i.e., the extra bits required when encoding with an approximate distribution q(x) instead of the true distribution p(x).

KL divergence is the additional bits required to compress data when using the wrong distribution q instead of the true distribution p.

Properties

Non‑negativity : KL divergence is always ≥ 0 and equals 0 only when the two distributions are identical.

Asymmetry : The order of the distributions matters; generally D_KL(P||Q) ≠ D_KL(Q||P).

Example

Consider two uniform distributions p(x) on interval Δ_p and q(x) on interval Δ_q. Their densities are constant within their respective intervals and zero outside. Substituting these densities into the KL formula yields

If Δ_p ≤ Δ_q the divergence is finite; otherwise, when Δ_p > Δ_q the divergence becomes infinite because q(x)=0 for x>Δ_q. For a concrete case with Δ_p=2 and Δ_q=3, the computed KL value is

Code implementation

A simple Python example uses scipy.stats to compute KL divergence for two discrete distributions representing grade frequencies in two schools.

import numpy as np
from scipy.stats import entropy

# Grade frequency vectors for schools A and B
p = np.array([0.30, 0.25, 0.20, 0.15, 0.10])  # School A (more A and B grades)
q = np.array([0.10, 0.15, 0.20, 0.25, 0.30])  # School B (more C and D grades)

kl_A_B = entropy(p, q)   # D_KL(A||B)
kl_B_A = entropy(q, p)   # D_KL(B||A)
print('KL(A||B):', kl_A_B)
print('KL(B||A):', kl_B_A)

The resulting KL values are shown below.

Applications

In variational autoencoders (VAEs) KL divergence appears as a regularization term that forces the approximate posterior Q(z|x) to stay close to the prior P(z) (usually a standard normal distribution), enabling effective latent‑space learning.

In reinforcement learning, KL divergence constrains policy updates (e.g., PPO) to limit the deviation between the new and reference policies, ensuring training stability. Recent models such as DeepSeek R1 adopt a variant called Group Relative Policy Optimization (GRPO), which uses KL as a regularizer to balance exploration and exploitation while avoiding extra value‑function computation.

Because KL divergence is asymmetric, alternative divergence measures have been developed to address its limitations.