Memory Optimization in Long Connection Services: A Three-Year Journey
Over three years, Meitu’s long‑connection service “bifrost,” built in Go with MQTT, cut memory use from 22.3 GB to 5.1 GB for 210 k connections—and about 10 GB for a million—by tuning buffers, reducing allocations, employing memory pools, and replacing a goroutine‑per‑connection model with an epoll‑driven event architecture.
This article details the three-year journey of Meitu's long connection service in memory optimization. The service, internally codenamed 'bifrost', was built in 2016 using Go and MQTT protocol. The architecture consists of seven modules: conf, grpcsrv, mqttsrv, session, pubsub, packet, and util.
The article explains Go's memory management system, including its garbage collection mechanism and memory statistics parameters such as go_memstats_sys_bytes, go_memstats_heap_inuse_bytes, go_memstats_heap_idle_bytes, and go_memstats_stack_sys_bytes. It highlights the challenges of memory consumption in long connection services, particularly with goroutine stacks and garbage collection.
The optimization journey is presented in three phases: business optimization, Go code optimization, and network model optimization. Business optimization involved adjusting buffer sizes and collection thresholds. Go code optimization focused on reducing temporary object allocation, using memory pools, and optimizing data structures. Network model optimization transitioned from the traditional goroutine-per-connection model to an epoll-based event-driven model.
Results show significant memory reduction: from 22.3GB to 5.1GB for 210,000 connections, and approximately 10GB for 1 million connections. The team plans to open-source the bifrost project and continue exploring network links, service capabilities, and storage optimization.
Meitu Technology
Curating Meitu's technical expertise, valuable case studies, and innovation insights. We deliver quality technical content to foster knowledge sharing between Meitu's tech team and outstanding developers worldwide.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.