How to Prevent Duplicate Requests on the Server Using Redis and Request Hashing

Some user requests may be sent repeatedly; while read‑only queries are harmless, duplicate write operations (e.g., order placement) can cause serious problems.

Typical duplicate scenarios include hacker replay attacks, client‑side resubmission, gateway retransmission, etc.

This article focuses on server‑side deduplication using a unique request identifier and does not cover client‑side click‑blocking.

Deduplication with a unique request ID

If each request carries a unique ID, Redis can be used to store the ID with a short TTL. When the key already exists, the request is considered duplicate.

String KEY = "REQ12343456788"; // request unique ID
long expireTime = 1000; // 1000 ms expiration, duplicate within 1 s is considered duplicate
long expireAt = System.currentTimeMillis() + expireTime;
String val = "expireAt@" + expireAt;
Boolean firstSet = stringRedisTemplate.execute((RedisCallback<Boolean>) connection ->
    connection.set(KEY.getBytes(), val.getBytes(),
        Expiration.milliseconds(expireTime),
        RedisStringCommands.SetOption.SET_IF_ABSENT));
final boolean isConsiderDup;
if (firstSet != null && firstSet) {
    isConsiderDup = false; // first access
} else {
    isConsiderDup = true; // key already exists, duplicate
}

Deduplication based on business parameters

When a unique request ID is unavailable, combine user ID, method name, and request parameters to form a key. For JSON parameters, sort keys, concatenate, and compute an MD5 hash to keep the key short.

String KEY = "dedup:U=" + userId + "M=" + method + "P=" + reqParamMD5;

If the request contains time‑related fields, exclude them before computing the MD5 so that rapid repeated clicks are still detected.

Helper class

public class ReqDedupHelper {
    /** Compute MD5 of request JSON after removing specified keys */
    public String dedupParamMD5(final String reqJSON, String... excludeKeys) {
        String decryptParam = reqJSON;
        TreeMap paramTreeMap = JSON.parseObject(decryptParam, TreeMap.class);
        if (excludeKeys != null) {
            List<String> dedupExcludeKeys = Arrays.asList(excludeKeys);
            if (!dedupExcludeKeys.isEmpty()) {
                for (String dedupExcludeKey : dedupExcludeKeys) {
                    paramTreeMap.remove(dedupExcludeKey);
                }
            }
        }
        String paramTreeMapJSON = JSON.toJSONString(paramTreeMap);
        String md5deDupParam = jdkMD5(paramTreeMapJSON);
        return md5deDupParam;
    }
    private static String jdkMD5(String src) {
        try {
            MessageDigest messageDigest = MessageDigest.getInstance("MD5");
            byte[] mdBytes = messageDigest.digest(src.getBytes());
            return DatatypeConverter.printHexBinary(mdBytes);
        } catch (Exception e) {
            return null;
        }
    }
}

Test logs show that without excluding the requestTime field the MD5 values differ, while excluding it yields identical hashes, confirming the deduplication logic.

Conclusion

The complete solution combines a unique request ID path (using Redis SETNX with expiration) and a business‑parameter path (MD5 of sorted JSON after removing volatile fields) to reliably prevent duplicate write requests.