What We Learned After a Year of Tinker: Android Hotfix Challenges & Solutions

2016‑03‑10 Tinker project launched and was open‑sourced at the MDCC conference on September 23 of the same year. After a year, two developers working half‑time filled many pits, achieved modest results, and faced criticism.

Will Tinker worsen Android's already problematic ecosystem and pose greater security challenges?

We aimed to let developers upgrade quickly with minimal cost, reflecting China's fast‑iteration demand. Over the year we encountered many pitfalls and filled many. This article shares the problems we faced and our solution approaches.

Tinker's Current Status

First, a brief overview.

Open‑source status

Tinker's repository: https://github.com/Tencent/tinker. It was Tencent's first open‑source project on GitHub, making Tencent rank first on GitHub weekly. We continue using Tinker in WeChat and promise the same version to external developers.

More than 60 of the top 1000 apps in YingYongBao use Tinker; over 1000 apps have integrated it via third‑party platforms.

Ecosystem status

Beyond developer count, Tinker has built a small ecosystem.

1. Hot‑fix service platform

Hot‑fix is not a trivial task; misuse can cause bigger problems, so integration can be difficult.

Hot‑fix is not a free lunch.

Some products want a complete service for both client and backend, requiring just one line of code for quick integration.

One line of code, fast integration.

Current solutions like TinkerPatch and Bugly provide one‑stop hot‑fix services based on Tinker, reducing developers' work.

2. OEM partners

Thanks to WeChat's influence, we have close ties with OPPO, Vivo, Huawei, Xiaomi, OnePlus, Lenovo, etc. They help solve compatibility issues and test upgrades. Their involvement also makes OEMs consider hot‑fix impact when customizing Android.

We oppose OEMs customizing WeChat for optimization; we prefer solving issues within Tinker so user experience stays consistent.

3. Code protection

Many developers need code protection. In version 1.7.0 we supported it via QZone fallback, but market implementations varied and were black boxes, so we dropped support in 1.7.6.

Recently we collaborated with Lejia, 360, iJiami, etc., to define a protection scheme compatible with hot‑fix, with rules:

Do not pre‑load classes.

When compiling oat on ART, inline must be disabled.

We aim for a healthy ecosystem where other hot‑fix solutions are also supported. Version 1.7.9 works with the above OEM protectors.

Derived open‑source projects include tinker-dex-dump, tinker-manager, TinkerPatch, etc.

The Road Walked Kneeling

An open‑source project is more than code; it requires communication with third‑party platforms and OEMs, but technology is the biggest factor. We strive for quality.

Below is a brief review of representative issues we faced over the year, hoping the problem‑solving process inspires you.

1. Qzone scheme issues on Dalvik and ART

The Qzone scheme caused major challenges, prompting the creation of Tinker. Detailed analysis can be found in “WeChat Android Hot‑Patch Practice Evolution”.

Example: a crash on Android 5.0 due to a static Boolean being null.

Code snippet: static Boolean addFriendTipsShow = false We added logs, inspected source, and compiled a custom ROM to dump memory addresses, discovering a wrong reference to static ImageView sightChangeImage.

Lesson: understand both the “what” and “why” before using a solution.

2. Android N mixed‑compilation issue

Resolved in Tinker 1.7.0. The issue originated from a crash reported by Huawei during Android N internal testing.

Root cause: Android N enabled JIT and mixed‑compilation by default, affecting hot‑patches.

Solution steps:

Read source code; reproduced locally by generating a full base.art.

Replace ClassLoader; this impacted startup time.

Later Android O moved ClassTable pointers upward, prompting alternative mitigations.

3. ART inlining causing crashes

Fixed in Tinker 1.7.6. A crash similar to the earlier address‑offset issue was traced to method inlining.

We abandoned per‑platform synthesis and switched to full‑coverage patches, which introduced OTA‑related startup slowdown and ANR issues.

4. Huawei WeChat dual‑instance crash

Found in Tinker 1.7.7. Certain Huawei devices crashed when using WeChat’s dual‑instance feature because the patch path wasn’t mapped, leading to security‑mode removal.

After reporting to Huawei, the issue was fixed in a new firmware.

5. Oppo/Vivo asynchronous dex2oat issue

Discovered in Tinker 1.7.6, resolved in 1.7.7. OEMs used async dex2oat, causing false assumptions that oat files were ready.

We coordinated with OEMs and added waiting logic during patch synthesis.

6. Dexdiff algorithm validation and performance

We provide analysis of the Dexdiff algorithm and MDCC slide. Validation steps include:

Generate 1000 random dex pairs and verify diff/patch.

Test with recent 100 WeChat versions.

Compile‑time verification of final dex legality.

We added 129 monitoring points; each change is analyzed online.

7. Pre‑Android N JIT issue

Fixed in Tinker 1.7.8. Some 6.0 devices crashed after patches due to JIT being unintentionally enabled.

Solution: filter out devices with JIT enabled before applying patches.

8. Odex corruption issue

Corrupted odex files caused NoSuchMethod crashes. We detect illegal ELF odex files (≈0.1% of users) and regenerate them.

9. OEM OTA‑induced startup latency

After system OTA, old oat files expire, causing long dex2oat on first launch, sometimes ANR. We explored multiple mitigations: version rollback, waiting dialogs, and asynchronous execution after OTA.

10. Resource‑related problems

Issues include OEM‑specific AssetManager implementations, checking dex/resource patch success, fast resource patch generation without unzip/zip, parsing resources.arsc, WebView reflection for mResDir and publicSourceDir, and a Xiaomi‑specific crash due to MiuiTypedArray cache.

Solution: clear TypedArray cache during resource patching.

11. Other issues

Xposed/pre‑load, classloader, ProGuard conflicts, etc.

We reflected on the challenges, especially after being criticized by Vivo, and refined OTA handling, dex2oat thread limits, and background patch synthesis.

Unfinished Work

Due to limited time, many technical details remain. We plan to address component proxy, startup protection, and welcome contributions.

Acknowledgments

Thanks to contributors from TinkerPatch, Baidu, Mogujie, UC, 58.com, GuoYing Tech, 360, Didi, Huawei, Vivo, Xiaomi, and many others.

References

Further reading includes articles on WeChat Android hot‑patch practice, Android N mixed‑compilation, Dev Club Tinker sharing, Tinker source code, Dexdiff analysis, ART inlining analysis, MDCC slides, and DexDiff tool.