Boosting Mobile Video Calls: Real-Time Low-Light Enhancement Using Spatial‑Temporal Constraints

Application Background

Most smartphones can capture video, but limited lens size and cost result in low photon flux per pixel, especially in indoor or low‑light scenes, causing under‑exposed and dark video streams that degrade real‑time mobile video calls. Low‑light video enhancement modifies pixel values to improve visual quality and expand usage scenarios.

Related Technologies

Existing low‑light video enhancement methods borrow from image enhancement techniques such as histogram equalization, contrast stretching, gamma correction, homomorphic filtering, tone mapping, and Retinex. These approaches either over‑enhance, amplify noise, lack adaptability, or demand excessive computation, making them unsuitable for real‑time mobile video.

Design Motivation

Current video enhancement solutions focus on single frames, ignoring inter‑frame correlation, which leads to inconsistent enhancement, flicker, and grain. Real‑time video chat also imposes strict limits on computational load and memory, rendering heavy algorithms impractical on mobile devices.

Our goals are to: (1) brighten low‑light video frames, (2) avoid flicker across consecutive frames, (3) preserve normal‑brightness frames without over‑enhancement, and (4) meet real‑time computational constraints.

Spatio‑Temporal Constrained Low‑Light Video Enhancement

We propose a real‑time algorithm that introduces spatial brightness and contrast constraints together with a temporal brightness consistency constraint. The optimization problem is convex and solved via a closed‑form least‑squares solution. Processing is performed in YUV420 space, affecting only the Y (luminance) channel while preserving UV chroma.

Brightness enhancement uses a learned function family F_I derived from paired low‑ and normal‑light video samples. To prevent contrast imbalance, we adaptively adjust the pixel value range, excluding extreme d% outliers, and apply a soft‑threshold method to generate a contrast enhancement function F_C. Temporal consistency is enforced by constructing a cost function G that penalizes brightness differences between the current frame and the average of neighboring frames, suppressing flicker.

The overall optimization combines three quadratic terms and is solved efficiently with least‑squares, yielding a low‑complexity, robust solution suitable for mobile devices.

Experimental Results

Subjective tests on various scenes (Bonsai, Face, Street, Kimino) show that our method outperforms four prior approaches, delivering brighter yet natural‑looking videos without flicker. Objective evaluations using MATLAB implementations demonstrate superior real‑time performance and lower CPU usage, as well as the highest NR‑CDIQA quality scores, especially preserving the appearance of already well‑lit frames.

Contributions

Offline training of a brightness enhancement function and adaptive contrast function for low‑light video.

Integration of spatial and temporal constraints into a unified regularized optimization framework.

Achieves real‑time processing with minimal computational overhead and strong robustness.

Conclusion

The proposed spatio‑temporal constrained enhancement algorithm significantly improves the subjective visual quality of low‑light video sequences while meeting real‑time mobile communication requirements. The method is computationally lightweight, robust, and has been patented in China, with a conference paper published at ISCAS 2017.