Understanding Continuous and Discrete Probability Distributions: Key Types and Relationships

Continuous and Discrete Probability Distributions

Probability distributions are divided into discrete (for random variables with a finite set of values, e.g., coin tosses, described by a probability mass function) and continuous (for variables that can take infinitely many values, described by a probability density function).

Mathematically, discrete distributions use summations while continuous distributions use integrals; for example, the expectation is computed accordingly.

Common discrete distributions include Bernoulli, Binomial, and Poisson. Common continuous distributions include Uniform, Exponential, and Normal.

Common Data Types

Data can be discrete (taking specific values, such as dice outcomes) or continuous (taking any value within a range, such as weight or height).

Types of Distributions

Bernoulli Distribution

The simplest discrete distribution with two outcomes, 1 (success) and 0 (failure). If X~Bernoulli(p), then P(X=1)=p and P(X=0)=1‑p. The expectation is E[X]=p and the variance is p(1‑p).

Binomial Distribution

Models the number of successes in n independent Bernoulli trials with success probability p. Its parameters are n and p. The mean is np and the variance is np(1‑p). Key properties: independent trials, two outcomes, fixed number of trials, constant success probability.

Poisson Distribution

Used for counting the number of events occurring in a fixed interval of time or space (e.g., calls in a call center). It assumes events occur independently and at a constant average rate λ. The mean and variance both equal λ.

Uniform Distribution

If X is uniformly distributed on [a,b], the probability density is constant 1/(b‑a). The mean is (a+b)/2 and the variance is (b‑a)²/12. The standard uniform case has a=0, b=1.

Exponential Distribution

Models the time between independent events occurring at a constant rate λ. The density is λe^{‑λx} for x≥0. The mean is 1/λ and the variance is 1/λ².

Normal (Gaussian) Distribution

Characterized by mean μ and variance σ², with the familiar bell‑shaped curve. It arises as the limit of sums of many independent random variables (central limit theorem). The density is (1/(σ√{2π})) e^{‑(x‑μ)²/(2σ²)}.

Relationships Between Distributions

Bernoulli and Binomial : A Binomial distribution is the sum of independent Bernoulli trials.

Poisson and Binomial : The Poisson distribution is the limiting case of a Binomial distribution when n→∞, p→0, and np=λ remains constant.

Normal and Binomial : The Normal distribution approximates a Binomial distribution for large n with np and n(1‑p) not too small.

Normal and Poisson : The Normal distribution approximates a Poisson distribution when λ is large.

Exponential and Poisson : If inter‑arrival times follow an Exponential(λ) distribution, the number of arrivals in a fixed interval follows a Poisson(λ t) distribution.

Reference

Machine Learning Beginner, https://mp.weixin.qq.com/s/2vQpN9OBFa7XvaxrYtGFgQ