Design and Implementation of Noise Reduction for JVM Sandbox Repeater Traffic Recording and Replay

1. Background

jvm‑sandbox‑repeater is an open‑source tool from Alibaba built on top of jvm‑sandbox that can dynamically enhance a target JVM and record/replay specified traffic. Two main problems were identified: (1) recorded traffic is sampled by a fixed rate, leading to many duplicate or similar requests that waste storage and replay resources; (2) result comparison is performed on the full response object, which is noisy for fields such as timestamps, random numbers, or newly added fields that should be ignored.

2. Requirements

Based on the background, the following enhancements are planned:

Result‑comparison denoising: add a configurable list of fields to ignore during comparison.

Traffic‑recording denoising and automatic conversion to test cases.

3. Design

3.1 Result‑Comparison Denoising

Add a new property List<String> ignoreFiled to RepeaterConfig so users can specify fields to be ignored. /** * Fields to be ignored during comparison */ private List<String> ignoreFiled = Lists.newArrayList();

Extend the comparator interface com.alibaba.jvm.sandbox.repeater.aide.compare.Comparable#compare with an additional List<String> ignoreFiled parameter. /** * compare to object * * @param left left object to be compare * @param right right object to be compare * @param ignoreFiled compare ignore some filed * @return compare result */ CompareResult compare(Object left, Object right, List<String> ignoreFiled);

Modify the MapComparator implementation to skip keys that appear in the ignore list. if (ignoreFiled != null && key instanceof String && ignoreFiled.contains(key)) { continue; }

Adjust the replay service ReplayServiceImpl#saveRepeat to retrieve the ignore list from configuration and pass it to the comparator. List<String> ignoreFiledLists = null; if (moduleConfigBORepeaterResult.getData() != null) { ignoreFiledLists = moduleConfigBORepeaterResult.getData().getConfigModel().getIgnoreFiled(); } CompareResult result = comparable.compare(actual, expect, ignoreFiledLists);

All other comparator calls must now provide the additional argument; passing null disables denoising for those calls.

3.2 Traffic‑Recording Denoising

3.2.1 Research

The existing repeater classifies and de‑duplicates traffic using seven dimensions: protocol, full Dubbo method signature, HTTP URL suffix, request/response length, request/response similarity, etc. Similarity can be measured by cosine similarity or edit distance, with thresholds to group similar requests.

3.2.2 Final Solution

The final architecture includes preprocessing of interface paths, similarity calculation (edit distance), a scheduled denoising job, and message‑queue based processing using Akka actors.

Key steps: path preprocessing, sub‑call detection, edit‑distance based similarity, scheduling, batch detail retrieval, message queue dispatch, and consumption.

Core code snippets:

public class AiDenoiseOfferJob extends QuartzJobBean {

    @Override
    protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
        // implementation omitted
    }
}

public class AkkaConfig {

    @Autowired
    private ApplicationContext applicationContext;

    @Autowired
    private ApolloConfig apolloConfig;

    @Bean
    public ActorSystem actorSystem() {
        return ActorSystem.create("my-actor-system");
    }

    @Bean
    public ActorRef myActor(ActorSystem actorSystem) {
        return actorSystem.actorOf(new RoundRobinPool(apolloConfig.getActorNum()).props(Props.create(MyActor.class, applicationContext)), "router");
    }
}

@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
    // Get denoise traffic details
    List
recordBOList = recordDenoise.getRecordDetailBOList(apolloConfig.getPageSize());
    int partitionSize = (int) Math.ceil((double) recordBOList.size() / apolloConfig.getActorNum());
    IntStream.range(0, recordBOList.size())
        .boxed()
        .collect(Collectors.groupingBy(index -> index / partitionSize))
        .values()
        .stream()
        .map(indices -> indices.stream().map(recordBOList::get).collect(Collectors.toList()))
        .forEach(t -> {
            myActorRef.tell(new MyMessage(t), ActorRef.noSender());
        });
}

public class MyActor extends AbstractActor {

    private final ApplicationContext applicationContext;

    @Autowired
    public MyActor(ApplicationContext applicationContext) {
        this.applicationContext = applicationContext;
    }

    @Override
    public Receive createReceive() {
        return receiveBuilder()
            .match(MyMessage.class, message -> {
                List
recordBOList = message.getRecordBOList();
                log.info("Received message size {}, thread {}", recordBOList.size(), Thread.currentThread().getName());
                RecordDenoise recordDenoise = applicationContext.getBean(RecordDenoise.class);
                if (!recordBOList.isEmpty()) {
                    long startTime = System.currentTimeMillis();
                    recordDenoise.doRecordDenoise(recordBOList);
                    long endTime = System.currentTimeMillis();
                    log.info("Processing time:{}", endTime - startTime);
                }
            })
            .build();
    }
}

4. Future Plans

Currently the system provides manual denoising for result comparison and automatic conversion of recorded traffic into test cases. Future work includes integrating JaCoCo‑based coverage analysis to map generated test cases to code paths and business scenarios, enabling more precise impact assessment.

Takeaway

If you find this design useful, consider following the public account "政采云技术" for more technical articles.