Estimating Coin Toss Probability with Maximum Likelihood in Python

Maximum likelihood estimation (MLE) is a widely used parameter estimation method. The intuitive idea is that if a random experiment yields a particular outcome, the experimental conditions are favorable for that outcome, implying a higher probability.

Assuming a random sample follows a known probability distribution but with unknown parameters, we estimate the parameters by observing many trials and selecting the parameter values that maximize the sample’s probability.

This article uses a simple discrete distribution example: estimating the probability of a head ("head") when tossing a coin. A coin toss is a classic Bernoulli experiment with outcomes head (1) or tail (0), described by the probability mass function:

For n independent tosses, the likelihood function is:

Using Python, we simulate the coin tosses and then employ the sympy library to express the likelihood function:

From the simulation, 53 heads and 47 tails are observed out of 100 trials, close to the assumed true probability of 0.5.

We then solve for the value of p that maximizes the likelihood using Python:

The result, unsurprisingly, is p ≈ 53/100 ≈ 0.5.

After taking the logarithm, the algorithm actually solves the equation:

This equation underlies the maximum likelihood estimation for logistic regression parameters.

To visualize the Bernoulli experiment, we plot the probability mass function centered at its mean (100 × 0.5 = 50):

The plot shows the discrete probabilities, with the maximum near 0.08.

In summary, this article applies maximum likelihood estimation to a simple discrete probability mass function using Python, and the same approach can be extended to continuous distributions by deriving the likelihood from the probability density function.