Enabling Near‑Local Remote Debugging for Java Apps on Youzan Cloud

1. Background and Introduction

1.1 Background

In the e‑commerce cloud, third‑party developers' applications run in containers managed by Youzan Cloud. For security and stability, the container execution environment must appear as a transparent black box to external developers. However, these apps depend on middleware such as Apollo, RDS, KVDS, making local debugging unrealistic, and direct JDWP remote debugging is prohibited. Therefore a remote debugging capability is required.

1.2 Overview

The Youzan Cloud Application Remote Debugging Tool provides a near‑local debugging experience without modifying application code or significantly affecting performance. It works by enhancing selected Java bytecode, recording execution snapshots when hit conditions are met, and allowing developers to replay snapshots via an IDE plugin.

2. Overall Architecture

2.1 Architecture Diagram

The diagram briefly describes the overall architecture:

IDE plugin handles state management, snapshot file management, and polls status to download and replay snapshots.

record‑server acts as the central scheduler, consisting of recording management, sandbox scheduling, and data management.

Recording management controls sandbox and recording state inside the container and dispatches IDE commands.

Sandbox scheduling provides executable sandbox commands inside the container.

Data management handles snapshot data requests.

Security policies restrict calls to only Youzan Cloud services.

Data and state flow use Kafka and Flink.

2.2 Design Considerations

To meet goals (non‑intrusive, minimal performance impact, reasonable snapshot size), the first phase made several trade‑offs:

Restore after recording: bytecode is enhanced on demand and restored after recording.

Recording scope: only classes in specified package paths are enhanced; standard libraries and third‑party packages are excluded.

Recording scenario: focus on single‑thread execution chains rather than full‑application snapshots to reduce overhead.

Hit condition: recording starts when the entry method is first reached and stops when it returns.

Line recording optimization: only lines with assignments or method calls are recorded.

Snapshot data optimization: only getters and public fields of objects are serialized.

Asynchronous processing: enhancement, snapshot persistence, and component cleanup are performed asynchronously.

3. IDEA Plugin

3.1 Plugin UI and Features

The plugin currently supports:

Permission management via phone number and token.

Remote debugging launcher: open application code, configure entry class and method, start debugging, trigger API calls, automatically jump to the first line of the entry after recording, and allow multiple debug sessions from a single recording.

3.2 Plugin Execution Flow

3.3 File Index Construction

The index is currently in‑memory; future versions will support local file search. The design reasons include:

Debug steps (resume, stepOver, stepInto, stepOut, call‑stack navigation) need line‑snapshot information.

Method information is displayed in the debug bar.

Class‑line snapshots enable call‑stack jumps.

Method‑call line snapshots support stepOver.

All line snapshots support stepInto.

pos records the current debugging position.

4. Application Method Execution Recording

Recording is built on a customized version of Alibaba’s open‑source jvm‑sandbox.

4.1 jvm‑sandbox Kernel Enhancements

Pseudo‑code of the enhancement logic is shown; the actual enhanced bytecode is displayed below.

Differences from the open‑source version

Extensions include:

Extended parameter lists for Spy.spyMethodOnCallBeforeExt, Spy.spyMethodOnCallReturnExt, Spy.spyMethodOnCallThrowsExt. Spy.spyMethodOnLineExt for line‑call notifications to record the current object, method parameters, and local variables.

4.2 Sandbox Recording Module Loading

4.3 Local Variable Table Analysis

Before bytecode enhancement, ClassMetaVisitor analyzes the class to obtain the local variable table.

appendLabel records label information.

appendLineNumber records line numbers and maps them to labels.

appendLocalVariable records local variable info and its start/end labels.

updateLocalVariableLine calculates the line range of a variable based on labels.

updateLocalVariableScope optimizes variable scope to avoid overlapping ranges.

registerMethodMeta stores the analyzed variable table.

4.4 Recording Process (Line Recording Example)

beforeLineExt obtains the computed local variable table, captures variable values, and persists them.

4.5 Recording Data Model

5. Sandbox Environment Management

The diagram shows sandbox installation and removal; record‑server interacts with the container through an ops proxy.

6. Future Plans

Enrich recording scenarios to handle more complex debugging problems.

Improve recording performance; current recordings significantly slow code execution and may cause timeouts.

Combine remote debugging with hot deployment (DCEVM + HotSwapAgent) for an even smoother development experience.