Exemplar Transformers Enable 8× Faster CPU‑Compatible Visual Tracking

In the paper *Efficient Visual Tracking with Exemplar Transformers*, a team from ETH Zurich proposes Exemplar Transformers, a new Transformer layer designed for real‑time visual object tracking. The layer is reported to be eight times faster than existing Transformer‑based trackers and can run efficiently on CPUs.

Key Contributions

Introduce Exemplar Attention, which reduces the quadratic cost of standard self‑attention by treating a small set of exemplar values as shared memory across dataset samples.

Integrate the Exemplar Transformer layer into a Siamese tracking architecture (E.T.Track), replacing the convolutional head without noticeable runtime overhead.

Demonstrate the first CPU‑real‑time Transformer‑based tracker.

Exemplar Attention Design

Inspired by the generalized “Scaled Dot‑Product Attention”, the authors redesign the attention operation based on two assumptions: (1) a small group of exemplar values can serve as shared memory among samples, and (2) a coarse query representation is sufficient to leverage these exemplars. This redesign reduces the number of feature vectors processed, yielding the reported speedup.

Integration into Siamese Tracker

The Exemplar Transformer layer replaces the convolutional head in the Siamese tracker E.T.Track. The added expressive power improves tracking performance and robustness while the impact on runtime remains negligible.

Benchmark Evaluation

The authors evaluate E.T.Track on six standard tracking benchmarks—OTB‑100, NFS, UAV‑123, LaSOT, TrackingNet, and VOT2020. The model achieves a 59.1% AUC, which is 2.2% higher than DiMP and 3.7% higher than the mobile version of LightTrack. Compared with the Transformer‑based tracker TrSiam, E.T.Track lags by only 2.2% in normalized precision (2.32%) and 3.12% in AUC, while delivering nearly an 8× speed increase on CPU.

Conclusions

The study shows that Exemplar Attention provides significant acceleration and cost reduction, and the Exemplar Transformer layer enhances the robustness of visual tracking models. The authors claim E.T.Track is the first Transformer‑based tracker capable of real‑time operation on computation‑constrained devices such as CPUs.