How to Make People Vanish in Real‑Time Using TensorFlow.js and MobileNet

Real‑Time Person Removal with TensorFlow.js

Jason Mayes, a senior web engineer at Google who focuses on IoT solutions, recently released a project called Real‑Time‑Person‑Removal . The demo runs entirely in the browser, uses TensorFlow.js, and can erase a person from a complex background in real time with roughly 200 lines of code.

Demo and Source Code

The source code is open‑sourced on GitHub and an interactive demo is hosted on CodePen. Users only need a computer with internet access and a webcam to try it.

Project repository: https://github.com/jasonmayes/Real-Time-Person-Removal

Demo: https://codepen.io/jasonmayes/pen/GRJqgma

The project quickly gained popularity, accumulating over 3.4k stars on GitHub.

Demonstration

The upper half of the video shows the original feed, while the lower half displays the result after person removal. Apart from occasional edge artifacts, the overall effect is impressive. In a showcase, Mayes placed a laptop playing a video on a bed; when his body covered the screen, the algorithm paused the background reconstruction and resumed it instantly after he moved away.

CodePen also provides a live example: click “Enable Webcam”, step back to let the algorithm capture the background, then return to see the “invisible” effect.

Project Mechanism

The core of the program relies on a pre‑trained MobileNet model provided by TensorFlow.js for person segmentation. MobileNet, introduced by Google in 2017 for mobile and embedded devices, uses depth‑wise separable convolutions to achieve a lightweight architecture, making it suitable for real‑time web applications compared to heavier models like YOLO or Fast‑RCNN.

const bodyPixProperties = {
  architecture: 'MobileNetV1',
  outputStride: 16,
  multiplier: 0.75,
  quantBytes: 4
};

After obtaining the segmentation mask, the algorithm computes the bounding box of the detected person and expands it by a scale factor of 1.3 (empirically chosen) to account for missed pixels.

// Calculate dimensions of bounding box.
var width = maxX - minX;
var height = maxY - minY;
var scale = 1.3;
var newWidth = width * scale;
var newHeight = height * scale;
var offsetX = (newWidth - width) / 2;
var offsetY = (newHeight - height) / 2;
var newXMin = minX - offsetX;
var newYMin = minY - offsetY;

The background is then updated pixel‑by‑pixel: pixels outside the scaled bounding box are restored from a stored background frame, while pixels inside are left transparent when a person is detected. If no person is found, the entire frame is refreshed.

// Update background outside the bounding box.
for (let x = 0; x < canvas.width; x++) {
  for (let y = 0; y < canvas.height; y++) {
    if (foundBody && (x < newXMin || x > newXMin + newWidth || y < newYMin || y > newYMin + newHeight)) {
      let n = y * canvas.width + x;
      data[n*4]   = dataL[n*4];
      data[n*4+1] = dataL[n*4+1];
      data[n*4+2] = dataL[n*4+2];
      data[n*4+3] = 255;
    } else if (!foundBody) {
      let n = y * canvas.width + x;
      data[n*4]   = dataL[n*4];
      data[n*4+1] = dataL[n*4+1];
      data[n*4+2] = dataL[n*4+2];
      data[n*4+3] = 255;
    }
  }
}
ctx.putImageData(imageData, 0, 0);

With this core logic, a single mouse click can make a person disappear from the video stream, reminiscent of a “snap” in popular culture.

Related Video‑Invisibility Projects

Mayes’s work is not the first to tackle video person removal. In 2019, the video‑object‑removal project demonstrated a similar capability by allowing users to draw a bounding box around an object, after which a model automatically tracked and hid the object throughout the video. That project leveraged SiamMask for tracking and Deep Video Inpainting for background reconstruction, both originating from CVPR 2019 research.

Project link: https://github.com/zllrunning/video-object-removal

While effective, the earlier approach sometimes produced artifacts such as misaligned lane markings after removing pedestrians. Interested readers are encouraged to run both projects and compare their results.