Building a Real-Time Video Transmission System with VisionSeed and Raspberry Pi

The article introduces how to build a complete video transmission (image‑transfer) system based on the VisionSeed AI camera module and a Raspberry Pi 4B. The author, Mao Jiangyun, a research engineer at Tencent Youtu Lab, explains the concepts, hardware, software stack, and practical steps.

1. Concept and Hardware

Video transmission means capturing video from a camera module and sending it in real time to another device for playback. VisionSeed is a small AI‑enabled camera module with a UVC camera and an infrared camera, while the Raspberry Pi 4B serves as the processing and streaming platform.

2. System Architecture

The system uses FFmpeg for encoding , an RTMP streaming server (either nginx‑rtmp or SRS), Wi‑Fi for network transport, and FFplay for client playback. The overall flow is: VisionSeed (UVC) → /dev/video0 on Raspberry Pi → FFmpeg encoding → RTMP server → client (FFplay).

3. Choosing an RTMP Server

The author compares nginx‑rtmp and SRS and shares personal experience. SRS provides lower latency (≈400 ms) compared with nginx‑rtmp.

4. FFmpeg Encoding

To capture the UVC video, run:

ffmpeg -i /dev/video0 -an -vcodec h264 -f flv rtmp://localhost:${port}/${api}

For hardware‑accelerated encoding on Raspberry Pi, use the OpenMAX driver:

ffmpeg -i /dev/video0 -an -vcodec h264_omx -f flv rtmp://localhost:${port}/${api}

Additional latency‑reduction options include:

-fflags nobuffer

-fflags flush_packets

-max_delay 10

-r 28 -video_size 1280x720 -g 50 -b:v 8192k

The final command used by the author is:

ffmpeg -r 28 -fflags nobuffer -fflags flush_packets -i /dev/video0 -vf fps=fps=28 -an -vcodec h264_omx -preset slower -tune zerolatency -max_delay 10 -r 28 -video_size 1280x720 -g 50 -b:v 8192k -f flv "rtmp://{RTMPIP}:{RTMPPORT}/live/1"

5. FFplay Playback

Typical playback command:

ffplay -i rtmp://192.168.1.1:2020/live/1

To eliminate the internal buffer and reduce latency, the author uses:

ffplay -autoexit -fflags nobuffer -fflags flush_packets -flags low_delay -noframedrop -strict very -analyzeduration 600000 -i rtmp://192.168.1.1:2020/live/1

Issues such as accumulated delay, freezing after stream loss, and frame‑rate instability are discussed, with solutions like using SRS or modifying FFplay source.

6. Optimizing Latency

Hardware acceleration via OpenMAX ( h264_omx ) is recommended. The author notes that the Raspberry Pi 4B can encode up to 1080p@30 fps using its dedicated codec module.

7. SRS Server Configuration

listen              1935

max_connections     1000

srs_log_tank        file

srs_log_file        ./objs/srs.log

vhost __defaultVhost__ {

min_latency     on

mr { enabled off }

mw_latency      100

gop_cache       off

queue_length    10

http_remux { enabled on; mount [vhost]/[app]/[stream].flv; hstrs on; }

}

These settings significantly improve real‑time performance.

8. Setting Up a Wireless Access Point on Raspberry Pi

The author follows the official tutorial to turn the Pi into a Wi‑Fi hotspot, providing a dedicated network for streaming. Key steps include updating apt sources, installing hostapd and dnsmasq , configuring static IP, DHCP range, and hostapd parameters.

Example /etc/apt/sources.list :

deb http://mirrors.tuna.tsinghua.edu.cn/raspbian/raspbian/ stretch main contrib non-free rpi

deb-src http://mirrors.tuna.tsinghua.edu.cn/raspbian/raspbian/ stretch main contrib non-free rpi

Example hostapd.conf :

interface=wlan0

country_code=CN

hw_mode=a

channel=149

wmm_enabled=1

auth_algs=1

wpa=2

wpa_key_mgmt=WPA-PSK

wpa_pairwise=TKIP

rsn_pairwise=CCMP

ssid=${YOUR_NETWORK_NAME}

wpa_passphrase=${YOUR_NETWORK_PSWD}

After these configurations, the Pi provides a stable AP, allowing clients to connect directly and receive the video stream with latency reduced from several seconds to about 400 ms.

9. Conclusion

The end‑to‑end solution combines VisionSeed, Raspberry Pi, FFmpeg, an RTMP server (preferably SRS), and a custom Wi‑Fi AP to achieve low‑latency real‑time video transmission. The author invites readers to share further optimization ideas.