Comparison of iOS Video Streaming Solutions for Remote Control

iOS remote control relies heavily on video output; this article evaluates three current iOS video streaming approaches—WebDriverAgent MJPEG image server, Apple ReplayKit, and QuickTime‑based QVH—by implementing prototypes, measuring latency, stability, and functionality, and summarizing the results.

Solution analysis

1. WebDriverAgent MJPEG image server : Uses the open‑source STF project to observe WDA image service video stream. Deployed on a machine, measured delay around 200 ms with noticeable stutter. Drawbacks include a relatively long service startup and lack of audio support.

2. ReplayKit video stream : Apple’s ReplayKit framework provides real‑time video capture via the built‑in screen recording component. It offers fast transmission but consumes significant device resources, causing heating and preventing simultaneous use of other live‑streaming apps. Implemented a Python socket server to receive the stream; observed latency of roughly 100‑200 ms.

3. QVH video stream : Utilizes the QuickTime component on macOS (and Linux) to record the iOS screen, an open‑source project available on GitHub. The stream is sent via WebSocket; measured latency around 200 ms, with overall end‑to‑end delay ranging from 200‑700 ms, which may affect user experience in remote‑control scenarios.

Partial code implementation (ReplayKit sample)

- (void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType {
    @synchronized(self) {
        switch (sampleBufferType) {
            case RPSampleBufferTypeVideo:
                // Handle video sample buffer
                if (!CMSampleBufferIsValid(sampleBuffer)) {
                    return;
                }
                if (tempVideoTimeStamp && (CFAbsoluteTimeGetCurrent() - tempVideoTimeStamp < 0.01)) {
                    NSLog(@"load frame...");
                } else {
                    [imageHandler pushOneFrame:sampleBuffer];
                    tempVideoTimeStamp = CFAbsoluteTimeGetCurrent();
                }
                break;
            case RPSampleBufferTypeAudioApp:
                // Handle audio sample buffer for app audio
                break;
            case RPSampleBufferTypeAudioMic:
                // Handle audio sample buffer for mic audio
                break;
            default:
                break;
        }
    }
}

Conclusion

The comparison table summarizes latency, stability, functionality, and system support for each method:

Latency : WDA ~200 ms, ReplayKit 100‑200 ms, QVH 200‑700 ms.

Stability : WDA prone to disconnection, ReplayKit stops when the device is locked or in a call, QVH depends on QuickTime running correctly.

Functionality : All support video streaming; ReplayKit and QVH also support audio.

System support : WDA and ReplayKit require macOS, while QVH works on macOS and Linux.

References

WebDriverAgent: https://github.com/facebookarchive/WebDriverAgent

STF project: https://github.com/DeviceFarmer/stf

ReplayKit documentation: https://developer.apple.com/documentation/replaykit

QVH project: https://github.com/danielpaulus/quicktime_video_hack