Implementing System-wide Screen Recording with ReplayKit on iOS

In mobile client development, there are scenarios that require recording user actions inside an app or even across the entire system, such as online proctoring, bug feedback, and game live streaming. Apple provides the ReplayKit framework to meet these needs, and the LOOK live client of NetEase Cloud Music uses it for cross‑app screen‑recording live streams.

ReplayKit History

ReplayKit first appeared in iOS 9 with a ReplayKit Extension that enables in‑app recording. The key classes are RPScreenRecorder (the recording task manager) and RPPreviewViewController (visual feedback). WWDC session “Going Social with ReplayKit and Game Center” introduced these APIs.

iOS 11 added system‑wide broadcast capabilities (ReplayKit 2). When the user starts screen recording from Control Center, the extension can capture the whole system screen and all audio, including microphone input, without stopping when the user switches apps. This is covered in the WWDC session “Live Screen Broadcast with ReplayKit”.

Since iOS 15, ReplayKit supports a Loop Buffer that continuously records up to 15 seconds and can be exported on demand, useful for capturing highlights without creating an extension. See the WWDC session “Discover rolling clips with ReplayKit”.

System‑wide Recording Flow

Prepare the app (e.g., start a live stream).

User launches ReplayKit from Control Center.

The ReplayKit Extension receives video and audio streams and pushes them to the server.

User stops recording from Control Center, ending the flow.

Creating and Integrating a ReplayKit Extension

In Xcode 14.1, create a new target and select “Broadcast Upload Extension”. Disable the default UI Extension option because system‑wide recording does not need a UI.

The generated files are SampleHandler.h and SampleHandler.m . The latter contains the lifecycle callbacks:

- (void)broadcastStartedWithSetupInfo:(NSDictionary<NSString *,NSObject *> *)setupInfo {
    // User requested to start the broadcast.
}

- (void)broadcastPaused {
    // User requested to pause the broadcast.
}

- (void)broadcastResumed {
    // User requested to resume the broadcast.
}

- (void)broadcastFinished {
    // User requested to finish the broadcast.
}

The extension receives three types of sample buffers:

RPSampleBufferTypeVideo : full‑screen video frames.

RPSampleBufferTypeAudioApp : audio generated by the system/app.

RPSampleBufferTypeAudioMic : microphone audio when the user enables it.

Processing them is done in processSampleBuffer:withType: :

// Audio‑video callback
- (void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType {
    switch (sampleBufferType) {
        case RPSampleBufferTypeVideo:
            // Handle video sample buffer
            break;
        case RPSampleBufferTypeAudioApp:
            // Handle app audio sample buffer
            break;
        case RPSampleBufferTypeAudioMic:
            // Handle mic audio sample buffer
            break;
        default:
            break;
    }
}

Practical Experience in LOOK Live

Within the extension we integrate only essential features to keep memory usage low (≈20 MB, about half of the 50 MB limit): network heartbeat, IM long‑connection, optional local push, and a Local Socket + AppGroup mechanism for data sync with the host app.

Communication between the host app and the extension uses two methods:

Local Socket for real‑time data exchange.

AppGroup for shared UserDefaults.

Each method alone has drawbacks (complexity for persistence with sockets, polling overhead with AppGroup), so we combine them: after updating UserDefaults, the extension notifies the host via socket.

Guiding Users to Start Recording

ReplayKit 2 requires users to open Control Center and long‑press the screen‑record button, which is cumbersome. Starting with iOS 13, RPSystemBroadcastPickerView can be placed in the UI to invoke the system broadcast picker directly. The following code programmatically triggers the picker:

if (@available(iOS 12.0, *)) {
    RPSystemBroadcastPickerView *picker = [[RPSystemBroadcastPickerView alloc] initWithFrame:CGRectMake(0, 0, 1, 1)];
    NSString *bundleId = [NSBundle thisBundle_bundleId];
    picker.preferredExtension = [bundleId stringByAppendingString:@".broadcast"];
    picker.showsMicrophoneButton = YES;
    for (id subview in picker.subviews) {
        if ([subview isKindOfClass:[UIButton class]]) {
            [(UIButton *)subview sendActionsForControlEvents:UIControlEventTouchUpInside];
        }
    }
}

Privacy Protection

Because system‑wide recording captures everything on screen, sensitive information (e.g., verification codes, contacts) could be exposed. LOOK Live implements a “privacy mode” that discards video frames and instead streams a static placeholder image while still allowing optional audio. The mode can be triggered automatically when entering privacy‑sensitive screens or manually via a UI button.

Challenges and Tips

The extension is limited to 50 MB of memory; exceeding this causes the system to kill the extension. Frequent memory spikes occur when network latency slows down streaming, causing buffered frames to accumulate. Recommendations:

Monitor memory and drop frames when a threshold is reached.

Offload heavy audio/video processing to the host app via Local Socket, keeping the extension only responsible for encoding.

Use proper error reporting when calling finishBroadcastWithError: so the system alert shows a clear reason.

- (void)networkingErrorNotificationHandler {
    NSError *error = [NSError errorWithDomain:@"replaykitDomain" code:1234 userInfo:@{NSLocalizedFailureReasonErrorKey : @"Network unavailable, please restart screen recording"}];
    [self finishBroadcastWithError:error];
}

By carefully managing memory, using real devices for debugging, and leveraging the host app for heavy lifting, developers can implement robust system‑wide screen‑recording features with ReplayKit.

Conclusion

ReplayKit has evolved from in‑app recording to full system broadcast and Loop Buffer capabilities. Despite its power, the activation flow remains complex, so clear user guidance and privacy safeguards are essential. With the techniques described above, developers can overcome the 50 MB memory ceiling and debugging challenges to deliver smooth screen‑recording experiences.