How Ant Group’s Paladin Engine Powered the 2022 AR New Year ‘Five Blessings’ Project

AR Gameplay Summary

In the 2022 New Year "Five Blessings" project, Ant Group introduced two AR experiences—AR writing blessings and AR fighting the Nian beast—to diversify gameplay, add a sense of technology, and align with the emerging XR market. The implementation was supported by the Oasis team, algorithm team, and the Paladin interactive engine.

Technical Metrics

Device coverage: 72%

Stability: 1 failure per 100,000 sessions

Launch time: Android 1.1 s, iOS 439 ms (instant start)

Reuse rate: A single Paladin solution powered AR fighting the Nian beast, AR airborne writing, handwriting playback, and video recording across both iOS and Android.

Key Challenges

1. Bitmap Vectorization

The first half of AR writing involves drawing a 2D character on a canvas, then converting the raster image into smooth, stretchable vector strokes, extruding them with thickness and rounded corners, and applying material and lighting. High‑resolution raster data caused performance issues on low‑end devices, so the computation was moved to C/C++ via Paladin’s JS binding, exposing a Canvas.toCurveData API that outputs GeoJSON paths, reducing vectorization time to 1 ms.

2. 2D‑to‑3D Transition Effects

Transitioning from orthographic to perspective camera rendering required aligning the two camera projections so that objects appear the same size at a specific depth. The process involved three stages: (1) orthographic 2D drawing and sticker placement, (2) AR perspective projection and 3D conversion, and (3) separating camera and model for additional effects. Mathematical derivations ensured consistent depth and scaling, allowing the 3D blessing to be positioned correctly behind the sticker.

3. Memory and CPU Optimization

Canvas resize support for different device resolutions.

ARSession camera resolution configuration (e.g., 1080p for 2K screens).

Video recording resolution reduction to save memory.

Request API supports ArrayBuffer for large 3D model data.

Optimized sticker upload methods.

Reduced unnecessary pipeline layers (intelligent, animation, container).

ARKit memory usage details (AppleCV3D ~16‑21 MB, CMCapture ~19‑23 MB, etc.) and thread‑level CPU consumption.

iOS Independent JSC Limitations

Using the standalone JavaScriptCore framework on iOS introduces differences in version, features, and performance compared to WKWebView. This impacts performance‑sensitive scenarios such as React Native and mini‑program workers. The lack of WASM support was mitigated by integrating a WAMR VM as an external module, enabling algorithm‑intensive AR/mini‑games to use WASM across platforms.

No JIT Issue

Because iOS independent JSC lacks JIT, all VMs (WAMR, quickjs, Hermes) run in interpreted mode. Solutions include using Hermes for React Native (limited) or employing WKWebView as a JS worker (bridge overhead). Ultimately, moving compute‑intensive code to WASM or awaiting Apple’s JSC updates is the viable path.

Paladin Interactive Engine

Paladin is a mobile‑first, Web 3D‑based, high‑performance, cross‑platform interactive engine targeting mobile AR and mini‑games. Its architecture combines web development ease with game‑engine performance, offering features such as:

Web 3D development model with low entry barrier.

Cross‑platform SLAM and spatial positioning (in progress).

Native camera and algorithm support for efficiency.

Adapter modules for third‑party engines (Cocos, White‑Lily, Laya) to publish games on Alipay mini‑games.

Business Architecture

Paladin bridges the gap between WebAR, native AR, and mini‑programs, providing a unified solution that leverages native camera, SLAM, and rendering while maintaining a web‑centric development workflow.

R&D Process

The development flow includes architecture design, iterative testing, and production pipelines visualized in the accompanying diagrams.

Limitations and Future Work

Current challenges include iOS independent JSC restrictions, lack of WASM, and the need for standardized XR APIs. Future plans involve aligning XR interfaces with WebXR standards, adopting HTML 6 camera standards, and enhancing 2D UI capabilities and realistic occlusion in the Oasis engine.