Estimating Coin Toss Probability with Maximum Likelihood in Python

Maximum likelihood estimation (MLE) is a common parameter estimation method. The intuitive idea is that if a random experiment has possible outcomes A, B, C… and outcome A occurs, the experimental conditions favor A, i.e., the probability P(A) is large. When a random sample follows a known distribution but its parameters are unknown, we estimate the parameters by observing outcomes and choosing the parameter that maximizes the sample probability.

This article uses a simple discrete distribution example to simulate a coin toss and estimate the probability of heads. A coin toss is a Bernoulli experiment with outcomes head (1) or tail (0), whose probability mass function is:

For n independent tosses the likelihood function is:

We now estimate p by maximizing this likelihood.

First we simulate coin tosses in Python:

Using the sympy library we write the likelihood function:

In a simulation of 100 Bernoulli trials we observed 53 heads and 47 tails, close to the assumed true value 0.5.

Next we solve for p that maximizes the likelihood using Python:

The result is unsurprising: 53/100 ≈ 0.5.

After taking logarithms, the algorithm actually solves the equation:

This equation underlies the MLE for logistic regression parameters.

To visualize the discrete probability mass function we compute the pmf centered at the mean (100 × 0.5 = 50):

The following plot helps understand the distribution:

Displaying all 20 discrete probabilities shows the maximum around 0.08.

The article demonstrates MLE for a simple discrete PMF using Python; the same approach applies to continuous distributions by deriving the likelihood from the probability density function. Readers are encouraged to run the code and compare results with R, SAS, and other tools to explore better MLE implementations.