How ByteDance’s STE Team Tackles Linux Kernel Memory Waste and Drives Cloud‑Native Innovation

ByteDance’s STE (System Technologies & Engineering) team, formed in 2015 and officially named in 2018, grew from a handful of engineers to a multi‑hundred‑person group with R&D centers worldwide, focusing on Linux kernel, firmware, compiler technology, virtualization, networking, and intelligent operations.

Young Team Behind Linux Kernel

The core kernel team, largely composed of post‑90s engineers, provides Linux kernel services for all internal ByteDance businesses, handling kernel management, process scheduling, virtualization, and networking. They emphasize nurturing young talent through mentorship and opportunities.

Solving Decades‑Old Memory Management Redundancy

Linux manages physical memory in 4 KB pages, even when using large pages (2 MB or 1 GB), leading to wasted struct page allocations. STE engineer Song Muchun led the HVO (HugeTLB Vmemmap Optimization) project to reduce this redundancy, potentially saving up to 16 GB per terabyte of server memory.

The year‑long effort (2020‑2021) involved code research, development, testing, and refactoring, ultimately gaining industry recognition from Huawei, Google, AWS, and Oracle.

Cloud‑Native Device Virtualization with VDUSE

To bridge the gap between container I/O limitations and traditional virtualization, STE created the VDUSE framework, enabling user‑space creation of virtual devices that integrate with the vDPA, virtio, and vhost subsystems. VDUSE was open‑sourced in 2020 and merged into Linux 5.15.

VDUSE devices are created via ioctl calls on /dev/vduse/control, exposing character devices for configuration and data exchange, and are now deployed at scale in ByteDance’s cloud‑native environment.

Community Engagement and Future Vision

The team stresses open‑source collaboration, iterating HVO through more than 20 versions based on community feedback, and aims to explore further cloud‑native and high‑performance networking scenarios. Their long‑term goal is to push foundational infrastructure forward while fostering a culture of mentorship and innovation.