Designing Adaptive Time‑Window Formulas: From Theory to Real‑World Cache Optimization

From tangled legacy code to hand‑crafted function graphs inspired by large models, the author reflects on the elegance mathematics brings to technical problems, arguing that the best solution is often an explanatory formula rather than a static number.

Pain of Unprovable Solutions

Faced with a legacy optimization task, the author needed to evaluate each step of the process to justify changes before deployment, yet the effectiveness could only be verified after release.

The requirement boiled down to finding usable cached data within an effective time window; if none existed, the window length would be extended. Determining the optimal extension length proved difficult because a fixed value could be either too short to be effective or too long to be meaningful.

This problem maps to a one‑directional, increasing time‑window model: initially the window grows slowly to capture recent data, then accelerates, with an upper bound beyond which data is irrelevant.

Encountering Formulas

Using a large language model, the author sought a formula that, given a time input, outputs a window length satisfying the "slow‑then‑fast with ceiling" behavior.

1. Linear Growth Formula

t: time interval (seconds, minutes, etc.)

k: scaling coefficient (e.g., k = 0.5)

W max : maximum window length to prevent unlimited growth

2. Exponential Growth Formula

W min : minimum window length (e.g., 0)

W max : maximum window length

k: growth rate coefficient (larger k → faster growth)

3. Sigmoid Smooth Transition Formula

t 0 : time at which the window reaches its midpoint

k: controls the steepness of the curve

The model provided six candidate formulas, illustrating how mathematical expressions can replace ad‑hoc parameter tuning.

Creating a Custom Formula

Combining the strengths of linear and exponential growth, the author devised a convex function with controllable minimum and maximum bounds:

W(t) = W_min + (W_max - W_min) * (t / T_max_effective) ^ p * k

where:

W(t): computed window length

t: elapsed time since a reference event

W_min: fixed minimum length (e.g., 5 s)

W_max: fixed maximum length (e.g., 3600 s)

p > 1: exponent controlling acceleration (p = 2 for quadratic growth)

k: scaling factor to fit the desired maximum at T_max_effective

T_max_effective: time at which the window should approach W_max (e.g., 600 s)

Determining Parameters

Set W_min and W_max based on business constraints.

Choose p to define the steepness of acceleration; p = 2 is a good starting point.

Calculate k so that when t = T_max_effective, W(t) ≈ W_max.

For the author's scenario: W_min = 5 s, W_max = 3600 s, T_max_effective = 600 s, and p = 2, yielding a smooth curve that starts slow and quickly reaches the ceiling.

Seeing the True Form

The finalized formula was plotted using an online graphing tool, revealing its characteristic convex shape (upper bound omitted for clarity).

Afterthoughts

While a handcrafted formula may seem excessive for a single requirement, it offers strong explanatory power and adaptability across scenarios, allowing the system to compute the optimal time window dynamically rather than relying on static thresholds.

The experience reinforced the beauty of mathematics—its precision, symmetry, and universality—transforming a routine engineering task into an elegant, provable solution.