Intuitive Explanation of the Exponential Weighted Moving Average Algorithm

In time‑series analysis we often need to predict future values based on past observations. A simple moving average can be insufficient because recent points should influence the forecast more strongly.

The exponential weighted moving average (EWMA) addresses this by giving the latest observation a higher weight and older observations exponentially decreasing weights. Its recursive definition is:

where vₜ is the EWMA at time t , θ is the current observation, and β (0 < β < 1) controls how much weight is given to the new observation versus the previous average.

v₀ is the initial value, often set to 0.

β close to 0.9 is typical in practice.

Unfolding the recursion yields an explicit series where the most recent observation has weight 1, the previous one β, the one before that β², and so on. Because βᵏ decays exponentially, older points quickly become negligible. The expanded form is shown below:

The effective window of influence can be approximated as t ≈ 1/(1‑β) . For β = 0.9, about ten recent observations dominate the average.

Mathematical insight : Using the second‑order limit, one can show that the weight of an observation decays to 1/e after t = 1/(1‑β) steps, confirming the intuitive window size.

Bias correction : When v₀ is set to 0, early EWMA values are biased low because the initial weight is disproportionately large. A simple fix is to initialize v₀ close to the first observation θ₁, but a more robust solution is to apply bias correction by dividing the raw EWMA by (1 ‑ βᵏ):

When β is close to 1, the denominator is small for early steps, amplifying the estimate and compensating for the lack of accumulated data. As k grows, the correction factor approaches 1, and the algorithm reverts to the standard EWMA, ensuring stable long‑term behavior.

In summary, EWMA provides a flexible, computationally cheap way to smooth time‑series data, with the β parameter allowing practitioners to tune the effective memory length. The bias‑correction technique resolves the common early‑stage underestimation problem, making EWMA suitable for both simple forecasting and as a component in advanced optimizers such as momentum methods in deep learning.