Uncovering the iOS 16 Keyboard Crash: Root Cause, Reverse‑Engineered Fix, and Zero‑Crash Deployment

🙋🏻‍♀️ The author, an Ant Group client engineer, discovered a critical iOS 16 keyboard bug that caused frequent crashes in the Alipay app (version 10.5.0.6000). By re‑implementing the system keyboard’s tryLock method in assembly, the crashes were completely eliminated.

Background

Ant Group’s technical challenge hero list invited engineers to solve tough problems. One entry reported a top‑priority iOS 16 keyboard crash (referred to as “Keyboard Crash”) that was hard to reproduce and had a large impact on online services.

Original Crash Information

Key crash log details include:

Incident Identifier: 7C53A274-4184-4E38-B27E-07B4E1335277
Hardware Model: iPhone13 4
OS Version: iPhone OS 16.6 (20G75)
Exception Type: SIGSEGV
Exception Codes: SEGV_MAPERR at 0x2ab3106e0
Crashed Thread: 0
0   libobjc.A.dylib   _objc_retain
...

The crash occurs when objc_retain accesses address 0x2ab3106e0, indicating a memory‑management error (SEGV_MAPERR).

Scale and Distribution

Daily PV of the keyboard crash stayed in the hundreds for over six months, appearing only on iOS 16 across all device models.

Information Summary

The stack top shows objc_retain and the crash distribution suggests the issue originates from iOS 16’s system keyboard memory‑management.

Analysis and Deduction

Key knowledge and tools used:

Software: Sublime Text, Xcode, lldb (commands: b, c, bt, frame select, di, image list, p/x, po, x/1b).

Assembly skills: Arm64 register description and instruction set.

Scripts: otool, custom fetch_class_text_from_all.sh.

Critical classes: UIKeyboardTaskQueue (keyboard core) and NSConditionLock (condition lock).

Dependency module: Ant’s DebugKit.framework.

1. Start from the Crash Point

By calculating the offset of the crashing function ( 0x00000001a5183a7c _objc_retain) within its binary image, the exact instruction address was identified.

2. Simulate the Scene

A device matching the crash log (iPhone 12 Pro Max, iOS 16.6) was connected to Xcode, breakpoints were set, and the function stack was reproduced, revealing that the direct cause was a wild pointer passed to objc_retain.

3. Full Investigation

Using DebugKit, all methods, properties, and ivars of UIKeyboardTaskQueue were exported. The class contains a private _deferredTasks (NSMutableArray) at offset 0x20 and a private _lock (NSConditionLock) at offset 0x10.

Read/write analysis of _deferredTasks showed six methods that read it and four that write it. Lock usage analysis revealed three lock methods and one unlock method.

4. Linking the Two Angles

All read/write methods acquire the lock once and release it once, but the method -[UIKeyboardTaskQueue continueExecutionOnMainThread] calls tryLockWhenReadyForMainThread and, when the lock fails, continues execution and later unlocks, breaking the lock pairing.

This lock‑failure path allows concurrent writes to _deferredTasks, producing a wild pointer that crashes when objc_retain is invoked.

5. Re‑simulation

A demo app reproduces the crash by forcing tryLockWhenReadyForMainThread to fail; the crash disappears when the lock succeeds.

6. Cross‑Version Comparison

Only iOS 16 exhibits the bug; iOS 15 and iOS 17 return early on lock failure, confirming that Apple fixed the issue in iOS 17.

Bug Report

The issue was submitted to Apple’s Feedback Assistant but has not yet been resolved for iOS 16.

Solution (In‑App Patch Source Code)

Two patch strategies were considered:

Rewrite -[UIKeyboardTaskQueue continueExecutionOnMainThread] to insert a conditional return after the lock attempt (requires modifying 95 assembly instructions).

Rewrite -[UIKeyboardTaskQueue tryLockWhenReadyForMainThread] to return early on lock failure, which is simpler and safer. This approach was chosen.

The patch consists of an assembly file fix_UIKeyboardTaskQueue.S that implements the early return logic, and a hook file fix_UIKeyboardTaskQueue.m that replaces the original method at runtime on iOS 16 arm64 devices.

Patch Implementation

#ifdef __arm64__
// fix_UIKeyboardTaskQueue.S
// Assembly implementation of early‑return for tryLockWhenReadyForMainThread
... (assembly code omitted for brevity) ...
#endif

#ifdef __arm64__
#import <UIKit/UIKit.h>
#include <objc/runtime.h>
@interface fix_UIKeyboardTaskQueue : NSObject @end
@implementation fix_UIKeyboardTaskQueue
+ (void)load {
    if (@available(iOS 16.0, *)) {
        NSString *systemVersion = [[UIDevice currentDevice] systemVersion];
        NSArray *verInfos = [systemVersion componentsSeparatedByString:@"."];
        if (verInfos.count >= 2 && [verInfos[0] isEqualToString:@"16"]) {
            class_replaceMethod(objc_getClass("UIKeyboardTaskQueue"),
                sel_getUid("tryLockWhenReadyForMainThread"),
                (IMP)fix_UIKeyboardTaskQueue_tryLockWhenReadyForMainThread,
                "B16@0:8");
        }
    }
}
@end
#endif

Effect of the Patch

After enabling the patch in Alipay App version 10.5.16.6000 (released 2023‑08‑25), daily crash PV dropped to zero. Overall crash PV across all versions fell by nearly 90%, and the issue is considered resolved.

Attachments

1. Patch source code (assembly and hook files). 2. Demo app source code showing the crash and its fix. 3. Script fetch_class_text_from_all.sh used to extract class implementations from the iOS binary. 4. Extracted iOS 16.6 assembly for the key methods.

Related Links

[1] Arm64 register description. [2] Arm64 instruction set reference. [3] NSConditionLock documentation.

💁🏼‍♀️ If you found this interesting, feel free to follow for more deep‑dive technical posts.