Optimizing a Fixed‑QPS Mock Interface with Thread Compensation in Java

In a previous post I described how to mock a fixed‑QPS interface using a Semaphore and basic thread‑safety techniques; the multithreaded test showed a QPS deviation within 5%, which I considered acceptable and did not further optimize.

After reading an open‑source article I was inspired to refine the implementation. The new approach creates a dedicated thread that continuously compares the theoretical request count with the actual count and uses a thread‑safe object to apply a compensating correction.

Core implementation:

package com.fun.moco.support

import com.fun.frame.SourceCode
import com.github.dreamhead.moco.ResponseHandler
import com.github.dreamhead.moco.handler.AbstractResponseHandler
import com.github.dreamhead.moco.internal.SessionContext
import com.github.dreamhead.moco.util.Idles

import java.util.concurrent.Semaphore
import java.util.concurrent.TimeUnit
import java.util.concurrent.atomic.AtomicInteger

import static com.google.common.base.Preconditions.checkArgument

/**
 * Fixed‑QPS interface handler
 */
class QPSHandler extends AbstractResponseHandler {

    private static final Semaphore semaphore = new Semaphore(1, true);
    /**
     * Access interval, controlled in microseconds
     */
    private final int gap

    private final ResponseHandler handler

    /**
     * Record the time of the first request
     */
    private long start

    /**
     * Count of processed requests; no thread‑safe class needed because increment occurs in a single thread
     */
    private int times = 0

    /**
     * Difference between actual and expected request counts; only positive values are handled because expected QPS is usually higher than actual
     */
    private AtomicInteger diff = new AtomicInteger(0)

    private QPSHandler(ResponseHandler handler, int gap) {
        this.gap = gap * 1000
        this.handler = handler
    }

    public static ResponseHandler newSeq(final ResponseHandler handler, int gap) {
        checkArgument(handler != null, "responsehandler cannot be null!");
        def handler1 = new QPSHandler(handler, gap)
        handler1.thread.start()
        return handler1;
    }

    /**
     * Actual implementation; uses microseconds for higher accuracy
     */
    @Override
    void writeToResponse(SessionContext context) {
        if (start == 0) start = SourceCode.getNanoMark()
        semaphore.acquire()
        if (diff.getAndIncrement() <= 0) Idles.idle(gap, TimeUnit.MICROSECONDS)
        times++
        semaphore.release()
        handler.writeToResponse(context)
    }

    /**
     * Periodically compute the gap between actual and expected request numbers and compensate missing requests using multiple threads
     */
    private Thread thread = new Thread(new Runnable() {
        @Override
        void run() {
            while (true) {
                SourceCode.sleep(30_000)
                long present = SourceCode.getNanoMark()
                def t0 = present - start
                def t1 = times * gap
                if (t0 - t1 > gap) diff.getAndSet((t0 - t1) / gap)
            }
        }
    })
}

Note that the times field is not protected by a thread‑safe class because the increment operation runs in a single thread; adding synchronization would actually increase QPS error.

The diff field is always positive in practice, indicating that the expected QPS is larger than the actual QPS. This discrepancy stems from the execution time of the response code and the inaccuracy of the Idles.idle() method.

Author: FunTester (original content published on the FunTester WeChat public account, original sharing enthusiast, recommended by Tencent Cloud and Juejin community, Zhihu Level‑7 author).