Tencent Shiply: Android Hotfix Evolution from VM Hook to Full Dex Replacement

Abstract

Online crashes, core function failures, and logic errors affect users immediately. Waiting for a new app version to pass store review and reach users extends the fault window. Hotfix shortens this gap from days to hours. However, making a patch run is only the start; the real challenge is delivering patches safely, stably, and controllably across diverse Android versions, chip architectures, vendor customizations, obfuscation, multi-Dex, resources, and native libraries.

1. Why Hotfix: The Gap Between Code Fix and User Relief

Mobile app fixes involve two distinct time segments: development time (locate, fix, test) and version time (repackage, submit, wait for review, release, wait for user upgrades). Code fixes often take hours, but traditional releases need 1 to several days. Hotfix can reduce patch release to hours or a day. For high-frequency crashes, payment failures, or unusable core screens, this time difference amplifies impact.

2. What Android Hotfix Actually Fixes

Java/Kotlin code compiles to Dex inside the APK. At startup, ClassLoader loads classes, and Dalvik/ART executes methods. A method call roughly goes through: find target class, find target method, jump to code entry. Hotfix intervenes on this chain to make the system execute new code.

3. Generation 1: VM Hook – Exploration and Limits

3.1 How VM Hook Replaces Code

In ART, each method has an ArtMethod object holding method info and entry point. AOT/JIT compilation, class preloading, and method inlining mean a patch cannot just load a new class. If the old class is in ClassTable or DexCache, ClassLoader may still find it. If the old method is compiled to native code or inlined, merely replacing the method may not change execution. VM Hook therefore performs a coordinated set of operations:

Place the fixed class in a patch Dex and make ClassLoader find it first.

Clear old class caches in ClassTable and DexCache.

Analyze callers potentially affected by method inlining.

Switch relevant methods to interpreter execution, bypassing AOT-generated native code.

This approach avoids pre-instrumenting all business code and has low runtime performance impact when the change scope is controllable.

3.2 Why VM Hook Became Harder

Its weakness stems from deep coupling with VM internals. ART's internal data structures, symbols, and compilation strategies evolve with system updates; vendors also customize. The framework must continuously adapt to clean caches, modify method entries, and handle inlining. When class inheritance, fields, or method structures change, the impact spreads along subclasses and call graphs. More complex patches mean more classes/methods to reprocess, increasing load time and compatibility risk. With APEX dynamic updates and cross-Dex inlining, judging runtime environment by Android version alone is insufficient. VM Hook proved hotfix can make single fixes lightweight, but left the problem of continuously chasing VM implementations.

4. Generation 2: Function Instrumentation – Bypassing VM Adaptation

4.1 A Fork in Every Method

Function instrumentation moves the jump point to compile time, pre-reserving a branch in every potentially patchable method. It analyzes and extracts changes at the final Dex bytecode stage. An instrumented method simplifies to:

if (patchIsAvailable()) { 
    return dispatchToPatch(); 
}
return originalMethod();

Without a patch, the original logic runs; after patch delivery, the app dynamically loads the patch Dex and forwards specific methods to new implementations. This uses only app-layer mechanisms, does not modify ArtMethod, and does not depend on ART private structures, yielding better system compatibility. If the patch does not depend on missed initialization timing or alter runtime data state, it can take effect immediately without an app restart.

4.2 The Real Challenge: Automatic Patch Generation

Inserting a branch is easy; the difficulty is accurately identifying changes between two builds and automatically generating a patch that can call the original code. From source to final Dex, desugaring, obfuscation, and R8 optimization occur. Function names, parameters, and access modifiers may change; some functions may be inlined or deleted. Even small business changes can produce extra differences in build artifacts. Instrumentation moves diff analysis to the final Dex bytecode stage to reduce interference from intermediate build steps, while customizing R8, reusing mapping and function IDs to keep two builds consistent. VM compatibility issues are eased, but the boundaries of method-level fixes become clearer: structural changes like modifying class inheritance, access modifiers, constructors, annotations are limited. Adding fields requires solving variable storage, object binding, and GC issues. Each instrumented method adds instructions; even with filtering third-party libraries, narrowing instrumentation scope, and performance modes, the package size and normal execution path still bear costs. For developers, patch code must follow more rules; complex syntax, R8 optimization results, or unrelated code differences increase build failure and debugging costs, which grow with code scale.

5. Generation 3: Full Dex Replacement – Shiply's Current Choice

5.1 Patches Are Not Just a New Dex

Previous generations worked on already-loaded methods: VM Hook changes entries, instrumentation adds a jump at the entry. Full Dex goes further: on the next app startup, let ClassLoader load the fixed code from the start. Shiply's current underlying solution is based on full Dex replacement, combined with large-scale business practice, continuously optimizing patch installation success rate, effect speed, system compatibility, and integration convenience. "Full" first means code is ultimately loaded as a complete Dex. Simultaneously, the solution supports resource and native library replacement. The three capabilities combined expand the update object from a single method to the app's code, resources, and native libraries, approaching a complete app content update.

Full Dex patch production starts from two APKs: the baseline APK running online, and the new APK containing fixes. The build tool compares Dex, resources, and native library differences, generating a smaller differential patch. On the device, the client uses the old Dex from the baseline APK and the differential data to resynthesize the fixed complete Dex, performs integrity verification and necessary optimization. On subsequent app starts, a new ClassLoader loads this code. Therefore, "full Dex replacement" describes the final loading form, not that the full Dex must be sent directly to users; network transmission can still be differential data.

5.2 Capability Boundaries: Beyond Fixes, Still Constrained by Release Rules

VM Hook centers on method entries, instrumentation on pre-embedded method jumps, full Dex replacement directly updates complete class definitions. Adding or modifying classes, fields, and methods can all enter this complete Dex; business writes patches as normal code changes. Beyond code, full Dex also supports resource and native library updates. Except for a few system-managed items like AndroidManifest.xml and RemoteView, most app content can be covered. The primary use remains fixing crashes and logic errors. Under controllable change scope, thorough verification, and compliance with release norms, it can also carry lightweight functional updates, combined with gray release to gradually expand coverage based on online data.

5.3 Synthesis and Restart Costs Can Be Engineered Down

Devices must complete differential synthesis, verification, and Dex optimization; patch installation time is usually higher than lightweight method patches. Since old classes cannot be unloaded once loaded, code patches typically take effect on the next startup. Newly synthesized Dex cannot fully reuse all preloading and optimization results from the original install; cold start and runtime performance need separate optimization. These costs concentrate in installation and startup phases, manageable via asynchronous installation, DexOpt, phased delivery, and performance tuning. Shiply's continuous optimization on top of full Dex targets exactly patch installation success rate, effect speed, system compatibility, and integration convenience.

6. Why the Evolution Went from Hook to Instrumentation to Full Dex

VM Hook first connected method entries with light runtime cost for function-level fixes. When system versions, vendor implementations, or inlining strategies changed, this deep-VM approach had to adapt. Instrumentation moved the jump to compile time; fixes no longer depend on ART private structures, compatibility improved significantly, and under conditions patches can take effect immediately. The fix target remained mainly method implementations. Class structure, fields, resources, and build consistency left costs in toolchains and development rules. Full Dex took another step: the device ultimately loads the fixed complete code; classes, fields, methods, resources, and native libraries all enter the same update; business writes patches as normal code changes. Synthesis and next-startup effect costs concentrate in the installation phase and have clear engineering means for continuous optimization. For apps with huge codebases, complex engineering systems, and wide device coverage, the solution must simultaneously answer: can it cover complex real-world modifications? Can it adapt to continuously changing Android systems and build toolchains? Can ordinary developers quickly and correctly produce patches? Can patches be fully verified and impact controlled when anomalies occur? Can the whole capability be maintained long-term for ordinary R&D teams? Full Dex satisfies these simultaneously, so hotfix engineering practice moved to this line: 1) loading mechanism from VM private implementation to more stable ClassLoader and complete code replacement; 2) fix target from replacing a method ASAP to prioritizing compatibility, fix completeness, and verifiable results; 3) product form from a single client SDK to a complete engineering system covering build, release, gray, and monitoring. This is why Shiply adopts full Dex and continues building platform capabilities on top of the underlying fix ability.

7. Tencent Shiply: From Low-Level Capability to Complete Hotfix Solution

An engine solves "how new code gets loaded" but cannot independently answer: Is the patch built on the correct online version? Do the patch changes match expectations? Who can release, and is there approval before release? Which users should be covered first, and how to gradually ramp up? Did the patch successfully download, install, and take effect? Did crashes and core metrics become abnormal after release? These questions together determine whether a hotfix can truly land safely. Shiply hotfix can be understood as four interlocking layers:

Low-level Fix & Access: Responsible for Dex, resource, and native library fix and loading, reducing integration cost for complex projects.

Build & Verification Layer: Responsible for baseline management, diff generation, artifact inspection, and compatibility validation.

Release & Gray Layer: Responsible for task management, approval control, and phased rollout.

Data Monitoring & Safety Governance Layer: Responsible for status sampling, stability observation, and risk detection.

7.1 Low-Level Fix & Access: Compatible with Complex Projects

Shiply's current underlying solution is based on full Dex replacement, combined with internal business practice, continuously optimizing patch installation success rate, effect speed, system compatibility, and integration convenience. Beyond the default integration, Shiply provides an Application proxy integration mode, reducing business project transformation cost and improving compatibility with dependency injection frameworks like Hilt. App hardening often changes APK package structure or code loading flow, potentially affecting conventional patch production and loading chains. For such scenarios, Shiply is compatible with mainstream hardening solutions and provides a dedicated patch production mode, enabling patches to build, install, and load correctly in hardened environments. Facing continuous changes in Android systems and build toolchains, Shiply continuously advances adaptation for system versions, AGP, and complex build scenarios, so businesses don't independently bear underlying framework maintenance costs.

7.2 Making Patch Build Results Verifiable

Hotfix relies on baseline APK, new APK, mapping, resource mapping, and other build artifacts. Any baseline selection error or build environment inconsistency may cause the patch to contain unexpected changes. Therefore, a safe hotfix process cannot stop at "build success"; it must verify: baseline version correctness; patch artifact completeness; actual changes matching fix intent; Dex, resource, and native library diffs within expected range; patch installability and loadability in target environments. Solidifying these experiences into tools and processes reduces reliance on individual expertise.

7.3 Controlling Release Radius with Approval and Gray Rollout

The faster the patch, the more restraint needed. Shiply provides patch task management, release approval, and gray rollout capabilities. Teams can first verify internally or with a small user group, then gradually expand coverage based on results, rather than exposing risk to all users at once. This mechanism's value is not just "one more approval button" but giving every patch release clear responsibility boundaries, operation records, and rollout rhythm.

7.4 Confirming Fixes with Data

Patch upload success does not mean the problem is solved; download success does not mean the fix logic has taken effect. Shiply collects key data around the patch lifecycle, helping teams observe download, installation, and effect situations, and analyze coverage performance by app version, system, or device dimensions. Release data can also link with stability monitoring and alerting. Teams not only see how many users the patch covered, but can further judge: whether original problem metrics decreased; whether new crashes or anomalies appeared; whether specific systems and models have anomaly concentrations; whether current data supports further gray expansion.

8. Large-Scale Practice: From Tencent Internal to Multi-Industry Adoption

8.1 Tencent Internal: Validated by Over 100 Apps

Currently, over 100 internal Tencent businesses have integrated Shiply hotfix, covering billion-level end users, with daily request volume reaching 10 billion. These businesses differ significantly in app scale, technical architecture, version cadence, and user scale, providing a complex verification environment. This accumulated practice has driven underlying solution optimization in patch installation success rate, effect speed, and system compatibility, and gradually precipitated platform capabilities like build verification, release approval, gray control, and quality monitoring.

8.2 External Services: Covering Banking, Internet, Automotive

Externally, Shiply hotfix serves multiple industries, typically: Banking, funds, and securities – rapid fix of core link anomalies and urgent logic errors, with changes auditable, gray-releasable, traceable; Internet apps – high-frequency crashes, function anomalies, lightweight feature gray, shortening problem recovery and new feature coverage cycles; Automotive and mobility – complex business link anomalies, system and model compatibility, and online faults affecting core experience. From long-term practice in Tencent's complex internal businesses to continuous validation in external multi-industry scenarios, Shiply hotfix is no longer just an underlying technical capability but a complete solution adaptable to different business requirements.

9. Mature Hotfix Is Systematic Risk Management

Shiply's underlying solution evolved from VM Hook, function instrumentation, to full Dex replacement. The current choice of full Dex replacement is because it is more complete and stable in complex businesses, large-scale devices, and long-term evolution. On top of that, access adaptation, patch building, approval, gray, data sampling, and stability monitoring form a closed loop, turning hotfix from an emergency technology mastered by a few experts into a stability infrastructure that R&D teams can continuously trust. Fast fixing is hotfix's value; fixing stably, releasing precisely, and observing clearly are the marks of hotfix maturity.