TCN Deep Dive: Causal & Dilated Convolutions for Long-Range Time Series Forecasting

01 Understanding the Problem

Predicting next-hour order volume requires historical context: the past hour, same hour yesterday, and same hour last week all matter. Standard convolutions would need excessive depth to reach 168 hours (one week) back. TCN solves this by combining causal convolution (only looks at current and past timesteps) with dilated convolution (skips timesteps to exponentially expand receptive field).

02 How Causal and Dilated Convolutions Work Together

Standard 1D convolution centered on the current timestep would access future data, causing leakage. Causal convolution restricts the kernel to current and past inputs only. With dilation rate d and kernel size k, the output at time t is: y_t = Σ_{i=0}^{k-1} w_i * x_{t - i*d} For example, with k=3, d=2, the output reads t, t-2, t-4 — never t+1. Typical TCN stacks residual blocks where dilation doubles each layer ( 1, 2, 4, 8, 16, 32). Each residual block contains two causal dilated convolutions (kernel size 3) with ReLU, plus a skip connection (identity or 1×1 conv) to ease gradient flow.

Receptive field calculation for L blocks, kernel size k, dilations d_l = 2^{l-1}: RF = 1 + (k-1) * Σ_{l=1}^{L} d_l With k=3 and 6 blocks ( d = 1,2,4,8,16,32): RF = 1 + 2*(1+2+4+8+16+32) = 253 hours — covering the 168-hour weekly lag. A 4-block version reaches only 61 hours, missing the weekly signal.

Key insight: Receptive field coverage only means the network can access distant history; whether it learns to use it must be verified experimentally.

03 Complete Case Study: Synthetic Order Forecasting

Data generation simulates ~125 days (3000 hours) of hourly orders with:

Daily cycle: sinusoidal peak at hour 9

Weekly cycle: +11 orders on weekends (dow ≥ 5)

Temperature: yearly + daily sinusoidal + noise

Promotions: 12-hour events every 110 hours starting at hour 80

Autoregressive dynamics:

orders[t] = 27 + 0.38*orders[t-1] + 0.16*orders[t-24] + 0.15*orders[t-168] + 0.30*daily[t] + 0.30*weekly[t] + 0.32*temperature[t] + 7*promo[t] + noise

Features: orders, temperature, promo, hour_sin/cos, dow_sin/cos (7 features). Lookback window = 192 hours. Train/val/test split by time (70%/15%/15%). Standardization uses only training-period statistics.

Models compared:

TCN-4 : 4 residual blocks, dilations 1,2,4,8 → receptive field 61h

TCN-6 : 6 residual blocks, dilations 1,2,4,8,16,32 → receptive field 253h

Seasonal naive : baseline using order value from 168 hours ago

Training: Adam (lr=0.002), MSE loss, gradient clipping (max_norm=1.0), 14 epochs, early stopping on validation loss. Batch sizes: train 64 (shuffled), val/test 128.

04 Evaluation & Analysis

Visualizations include:

Receptive field growth vs. weekly lag (168h) and TCN-4 limit (61h)

Training/validation loss curves to detect overfitting

Time-series plots of first 10 test days and scatter plots of all test predictions vs. actuals

Error sliced by hour-of-day and by absolute weekly change magnitude

Results are judged by test MAE and RMSE (lower better). The article emphasizes: do not conclude from receptive field size alone; rely on actual test metrics and error breakdowns.

05 Parameter Tuning Guidelines

Two primary knobs:

Number of dilation layers : Compute the critical business lag (e.g., 168h), ensure receptive field covers it. Adding more layers increases compute and overfitting risk without guaranteed gains.

Kernel size : Larger kernels aggregate more adjacent timesteps per layer, expanding receptive field faster but adding parameters. The example uses kernel=3, delegating long-range coverage to dilation.

TCN suits tasks with clear temporal ordering and multi-scale historical patterns (orders, sensor readings, traffic). Two strict practices:

Never leak future-known features into prediction inputs (e.g., tomorrow's weather).

Respect temporal order in data splits and standardization (fit scaler on training period only).

The case study predicts next hour using only features available up to the previous hour, illustrating the boundary.

Conclusion

TCN's core combination: causal convolution enforces temporal boundary, dilated convolution expands historical reach, residual connections enable deep training . Whether long-range dependencies are actually exploited must be validated on time-ordered test sets, not just by receptive field formulas. Suggested extensions: shrink input window below 168h to observe weekly-info loss; change target to 24-hour horizon and compare autoregressive vs. direct multi-step prediction.