How We Cut Video Detection Memory Usage by 78% with WebAssembly and WorkerFS

Background & Current Situation

The creator service platform provides a PC‑side video upload entry for community creators. As functionality expands, the volume of uploaded videos has surged, leading to frequent user complaints such as corrupted videos, blurry playback, exposure issues, and black screens with audio only.

These problems degrade user experience and risk losing loyal users. Since April 2024, the number of video cases has multiplied, prompting the need for a fast, reliable pre‑upload detection capability.

Industry Practices

Many platforms perform server‑side detection after the entire file is uploaded. While this can catch illegal content (e.g., pornographic or terror‑related videos), it has three major drawbacks:

Detection only starts after the full upload, causing long wait times for creators.

Extended posting latency harms creator experience.

Increased bandwidth costs.

Despite these issues, server‑side detection remains effective for content moderation.

Our Team’s Solution

We built a client‑side pre‑detection pipeline using C + ffmpeg compiled to WebAssembly via Emscripten. The WebAssembly module parses video metadata, validates AV packets, and checks for structural anomalies such as missing moov atoms, malformed NAL units, or absent video tracks.

Key detection steps:

Parse container metadata (size, bitrate, rotation).

Extract color‑space information (HDR, DP3, bit depth).

Identify source platform or editing tool via embedded metadata.

Sample detection flow diagram (image omitted for brevity).

Problems to Solve

During integration we discovered three critical issues:

Memory leak : Large video uploads (e.g., 800 MB) caused memory usage to spike to 3 GB and never release.

Large‑video detection failure : Files larger than ~800 MB triggered allocation errors.

Slow detection speed : Converting the entire file to an ArrayBuffer and copying byte‑by‑byte into WASM memory took ~14 s for an 800 MB file.

Memory Optimization Plan

Investigation revealed that the root cause was the WASM memory model: malloc‑based allocations could grow but never shrink, and the ArrayBuffer size limit (≈2 GB) capped the detectable video size.

We replaced the copy‑into‑memory approach with direct file access using avformat_open_input. To make a file path available inside the WASM sandbox we mounted the video into a virtual file system provided by Emscripten: WORKERFS – read‑only access to File and Blob objects inside a WebWorker.

Build command (attributes stripped):

emcc -O3 \
  -I ${FFMPEG_DIST_DIR}/include \
  -L ${FFMPEG_DIST_DIR}/lib -l avcodec -l avformat -l swresample -l avutil -l workerfs.js \
  -I ${CJSON_SOURCE_DIR} \
  -s EXPORTED_FUNCTIONS="['_get_video_info', '_extract_video_data']" \
  -s WASM=1 \
  -s ALLOW_MEMORY_GROWTH=1 \
  -s EXPORTED_RUNTIME_METHODS=["wasmMemory", "FS", "WORKERFS", "ccall"] \
  -fsanitize=address \
  -o ${workspaceFolder}/sociality/main.js ${workspaceFolder}/sociality/main.c \
  ${CJSON_SOURCE_DIR}/cJSON.c ${CJSON_SOURCE_DIR}/cJSON_Utils.c

After exporting WORKERFS, we mount the video file inside the worker, call avformat_open_input with the virtual path, and process the stream without extra memory copies.

Modified C function (excerpt):

/* extract_video_data now releases the received buffer directly */
void extract_video_data(uint8_t *data, int size) {
    // ... processing logic ...
    free(data); // immediate release
}

Pressure tests showed memory usage stabilizing around 900 MB for repeated uploads, compared to >3 GB with the previous SDK. Detection time for an 800 MB file dropped from ~95 s to ~20 s (≈78 % speed improvement), and a 2 GB file processed in ~61 s.

Summary

By integrating WebAssembly, WorkerFS, and direct ffmpeg file access, we eliminated costly memory copies, resolved out‑of‑memory crashes, and dramatically accelerated large‑video detection. The solution is now live on the creator platform, improving upload reliability and user experience.