Why Does Template Mode Outperform Connection Mode in Venus‑Cache? A Deep Dive

Introduction

When a domain uses Venus‑cache, it was observed that the Connection mode (caching and continuously using the connection in the Template) can yield higher performance than using the Template mode directly.

Two Access Modes

MemcachedTemplate.getConnectionFactory().getConnection().get("key", serializer) – Connection mode

MemcachedTemplate.get("key") – Template mode

A detailed performance test was conducted to analyze the differences.

Test Investigation

Test Scenario

Because some domains using Venus‑cache have been migrated to the cloud, pressure tests were performed on both physical machines and virtual machines. The test environment parameters are shown below.

Test Results

Physical machine (Template max connections 100):

Cloud host (Template max connections 20):

The results show common patterns across platforms:

Template throughput is greater than Connection.

Template can quickly reach the QPS peak.

When Template reaches a turning point, QPS drops sharply as thread count increases.

Connection QPS continues to rise with thread count and eventually surpasses Template.

Key Questions

Why does Template performance decline as thread count increases?

Why does Connection performance improve with more threads?

Why is Template throughput higher than Connection?

Report Analysis

The three questions are answered through source code analysis, JMC observation, and JMH micro‑benchmarks.

Background:

We use the SpyMemcached framework for Memcached, which differs from Jedis by adopting an NIO model.

In SpyMemcached, each object in the pool is a thread that continuously performs selector.select(). Thus, an object in the pool is essentially a thread, unlike traditional connection pools where objects are lightweight.

Question 1: Template vs Connection Throughput

Before reaching the CPU core count, Connection uses a single NIO thread and cannot fully utilize CPU resources, while Template can leverage multiple cores, resulting in higher throughput.

Question 2: Performance Trend with Thread Count

Template incurs additional overhead from Commons‑pool when borrowing and returning objects, requiring multiple locks per operation. As thread count grows, lock contention increases, causing performance degradation.

Micro‑benchmark results (JMH) show that the borrow/return operation’s lock wait count rises sharply with more threads:

Further JMC observation reveals that SpyMemcached’s LinkedBlockingQueue experiences increased wait counts, but the impact on QPS is limited.

Analysis of the underlying data structures shows:

LinkedBlockingDeque used by Commons‑pool has a single lock shared by producers and consumers.

LinkedBlockingQueue used by SpyMemcached has separate putLock and takeLock, allowing offer and drainTo operations to proceed without blocking each other.

Therefore, Commons‑pool’s multiple lock acquisitions and lack of lock separation cause severe performance drops as threads increase, while LinkedBlockingQueue’s lock design keeps performance stable.

Final Summary

Commons‑pool is slow because each borrow/return operation requires four lock acquisitions and does not separate put and take locks, leading to contention and context switches that reduce QPS. In contrast, LinkedBlockingQueue’s dual‑lock design avoids such bottlenecks.

Solution: use multiple Connection instances stored in an array and select one by round‑robin or threadId % size to distribute load and avoid the single‑connection bottleneck.