Understanding ARMA: The Core of Stationary Time Series Models

ARMA Time Series Overview

AR model (Auto Regressive)

MA model (Moving Average)

ARMA model (Auto Regressive Moving Average)

Let {X_t} be a zero‑mean stationary series that satisfies a linear relation with a zero‑mean white‑noise error {ε_t} of variance σ². An AR(p) process can be written as X_t = φ₁X_{t-1}+…+φ_pX_{t-p}+ε_t, where φ = (φ₁,…,φ_p) is the autoregressive coefficient vector.

Introducing the back‑shift operator B (B X_t = X_{t-1}) simplifies the representation. The operator polynomial Φ(B)=1-φ₁B-…-φ_pB^p allows the model to be expressed compactly as Φ(B)X_t = ε_t.

Similarly, an MA(q) process uses the moving‑average coefficient vector θ and can be written as X_t = θ₁ε_{t-1}+…+θ_qε_{t-q}+ε_t, or equivalently Θ(B)ε_t = X_t with Θ(B)=1+θ₁B+…+θ_qB^q.

An ARMA(p,q) model combines both: Φ(B)X_t = Θ(B)ε_t. When q=0 it reduces to an AR(p) model; when p=0 it reduces to an MA(q) model.

For a stationary ARMA model, the following assumptions are required:

The polynomials Φ(B) and Θ(B) share no common factors.

All roots of Φ(B)=0 lie outside the unit circle (stationarity condition).

All roots of Θ(B)=0 lie outside the unit circle (invertibility condition).