How We Enabled Live Streaming for WeChat Mini‑Games: Cross‑Process Video & Audio Capture

Background

WeChat mini‑games recently added one‑click live streaming via Video Channels, allowing users to start a broadcast directly from games like Jump Jump or Happy Landlord.

Because mini‑games run in an isolated process for performance and security, audio/video data must be transferred to the main process for streaming, presenting several challenges.

Video Capture & Streaming

Screen Recording?

The intuitive solution is to use the system screen‑recording API MediaProjection, which offers simple integration, reasonable stability, low performance impact (<10% FPS drop), and allows direct processing and streaming in the main process.

However, this approach was rejected due to the need for a system permission dialog, privacy concerns when the game is paused, and the requirement to display a comment widget that should be visible only to the streamer.

Mini‑Game Rendering Architecture

The rendering flow consists of MagicBrush (the game rendering engine) drawing to a SurfaceView ‑provided Surface. The mini‑game process does not render to the main process.

Recording Scenario

When recording, the output target switches from SurfaceView to a Renderer that creates a SurfaceTexture. The Renderer receives frames via onFrameAvailable, converts them to GL_TEXTURE_EXTERNAL_OES, and renders to another Surface.

Modified Recording Scheme

By replacing the MP4 encoder with the live‑streaming module, the same rendering pipeline can be reused for streaming. The key is to transfer the Surface from the mini‑game process to the main process via Binder, because Surface implements Parcelable and can be passed across processes.

The final architecture reduces the number of Renderer instances from three to one, improving performance and simplifying the pipeline.

Compatibility & Performance

Cross‑process rendering works on Android 5.1+ except for occasional black‑screen issues on Android 5.x. Frame‑rate impact is around 15% on average, with the main process’s CPU usage increasing while the mini‑game process’s CPU usage slightly decreases.

SurfaceView combines a surface and a view. SurfaceView's view components are composited by SurfaceFlinger (and not the app), enabling rendering from a separate thread/process and isolation from app UI rendering.

Audio Capture & Streaming

Solution Selection

Android 10+ provides AudioPlaybackCapture, but due to high API level requirements we opted to implement our own audio capture and mixing.

Capture‑Side Conditions

API 29+

RECORD_AUDIO permission

MediaProjection permission (shared with screen capture)

Configure AudioPlaybackCaptureConfiguration to include/exclude usages and UIDs

Captured‑Side Conditions

Player’s AudioAttributes usage must be USAGE_UNKNOWN, USAGE_GAME or USAGE_MEDIA

App’s CapturePolicy set to AudioAttributes#ALLOW_CAPTURE_BY_ALL Declare android:allowAudioPlaybackCapture="true" in the manifest for API 29+

Overall, we chose a custom capture solution to support devices below Android 10.

Cross‑Process Audio Transfer

Audio data must be transferred from the mini‑game process to the main process at ~40 ms intervals (≈8 KB per 16 ms). We evaluated Binder, LocalSocket, MMKV, SharedMemory, and Pipe. LocalSocket with an ObjectStream provided the lowest latency (≈1 ms on most devices) and acceptable stability.

Security considerations for LocalSocket include randomizing the socket name (e.g., using MD5 of AppId and user ID) and adding a handshake authentication step.

Multi‑Process Issues

glFinish Causing Frame‑Rate Drop

Profiling revealed that glFinish in the main‑process renderer blocked for over 100 ms when the mini‑game process performed heavy GL work. This caused severe frame‑rate drops on low‑end devices.

Removing the unnecessary glFinish call eliminated the bottleneck and aligned the main‑process frame rate with the mini‑game’s output.

Background Process Priority

Live streaming frames were limited to ~16 FPS because the encoding thread ran in a background cgroup with limited CPU share. Raising the thread’s real‑time scheduling policy with chrt increased the frame rate to ~25 FPS, while moving the process to the foreground raised it to 30 FPS.

Android’s android.os.Process.setThreadPriority only adjusts the nice value, which does not affect the cgroup limits. Properly setting the scheduling policy is required for significant improvement.

Resolution

We created a foreground Service during live streaming to keep the encoding thread in the foreground cgroup, ensuring sufficient CPU allocation.

Conclusion & Outlook

Cross‑process rendering via Surface and careful IPC selection solved the live‑streaming challenges for WeChat mini‑games. The experience demonstrates that multi‑process architectures can be viable when leveraging Android’s surface‑sharing capabilities, while also highlighting the importance of GL synchronization and process‑priority management.

Future work includes exploring in‑process streaming modules to further reduce latency and investigating more robust security mechanisms for Unix domain sockets.