How EWMA‑Based Dynamic Thresholds Cut False Alarms in Monitoring

Introduction

The traditional anomaly detection approach uses a fixed static threshold, which often leads to excessive false alarms for highly variable metrics like network traffic, increasing operational workload. To address this, the article proposes dynamic threshold methods based on machine‑learning concepts.

EWMA‑Based Detection

For time‑series data, recent values depend strongly on previous points. By fitting a curve to recent data and detecting deviations, anomalies can be identified. The Exponential Weighted Moving Average (EWMA) is used for curve fitting.

EWMA recursive formula:

EWMA(1) = p(1)  // initial value

EWMA(i) = α * p(i) + (1‑α) * EWMA(i‑1)  // α is the smoothing factor (0‑1)

Each EWMA value gives higher weight to more recent data. Using the 3‑sigma rule, values outside the confidence interval trigger alerts.

Advantages

Detects a secondary short‑term anomaly shortly after a primary one.

Responds quickly because it emphasizes recent points.

Highly sensitive when historical variance is low.

Disadvantages

Less effective for gradual anomalies.

May consider a prolonged anomaly as normal.

Business curves with regular spikes can cause misinterpretation.

High sensitivity can increase false positives as variance grows.

Periodic‑Based Detection

Many metrics exhibit daily cycles (e.g., VIP traffic). By collecting the same time‑of‑day values from the past 14 days, a reference set is built.

Static threshold logic using this reference:

def simultaneous(data, min_threshold, max_threshold):
    last_time = data.index[-1]
    last_time = datetime.datetime.strptime(last_time, "%Y-%m-%d %H:%M:%S")
    simultaneous_data = []
    for i in range(1, days_num+1):
        before_time = last_time + datetime.timedelta(days=-i)
        before_time_index = before_time.strftime("%Y-%m-%d %H:%M:%S")
        if before_time_index in data.keys():
            simultaneous_data.append(int(data[before_time_index]))
    if int(data[-1]) < min(simultaneous_data) * min_threshold:
        return "突降"
    if int(data[-1]) > max(simultaneous_data) * max_threshold:
        return "突增"

Choosing appropriate min/max thresholds is critical; using average values often works well.

Amplitude‑Based Detection

When absolute values differ greatly across days, comparing raw values fails. Instead, the method uses relative amplitude:

Amplitude at time t = (x(t) – x(t‑1)) / x(t‑1).

By comparing the current amplitude against the maximum absolute amplitude from the past 14 days, sudden spikes or drops are detected.

def amplitude_max(data, threshold):
    last_amplitude = 0.0
    last_time = data.index[-1]
    last_time = datetime.datetime.strptime(last_time, "%Y-%m-%d %H:%M:%S")
    last_amplitude_time = last_time + datetime.timedelta(minutes=-1)
    last_amplitude_time_index = last_amplitude_time.strftime("%Y-%m-%d %H:%M:%S")
    if last_amplitude_time_index in data.keys():
        last_amplitude = float((float(data.values[-1]) - float(data[last_amplitude_time_index])) / float(data.values[-1]))
    last_time = last_time + datetime.timedelta(days=-1)
    amplitude_data = []
    for i in range(0, days_num):
        now_time = last_time
        prior_time = last_time + datetime.timedelta(minutes=-1)
        now_time_index = now_time.strftime("%Y-%m-%d %H:%M:%S")
        prior_time_index = prior_time.strftime("%Y-%m-%d %H:%M:%S")
        if now_time_index in data.keys() and prior_time_index in data.keys():
            tmp = float((float(data[now_time_index]) - float(data[prior_time_index])) / float(data[now_time_index]))
            amplitude_data.append(abs(round(tmp, 2)))
        last_time = last_time + datetime.timedelta(days=-1)
    if abs(last_amplitude) > max(amplitude_data) * threshold and last_amplitude > 0:
        return "突增"
    if abs(last_amplitude) > max(amplitude_data) * threshold and last_amplitude < 0:
        return "突减"

Advantages

More sensitive than absolute‑value comparison.

Accounts for periodic trends, reducing false alerts from regular spikes.

Disadvantages

Requires a smooth underlying curve.

Periodic sharp drops must align precisely, otherwise false alerts occur.

Percentage‑based thresholds may misfire during low‑traffic periods.

Not all sharp drops indicate faults; they may stem from upstream fluctuations.

Ensemble Voting

The three methods—EWMA, periodic‑based, and amplitude‑based—each have strengths and blind spots. By applying a majority‑vote rule (at least two methods must flag an anomaly), the system achieves higher accuracy and lower false‑positive rates.

For even better precision, open‑source projects such as Skyline can be integrated to expand the algorithm library.