NorthStar Large‑Model Training Framework: Architecture, APIs, Pipeline and Multi‑GPU Strategies

The article introduces the NorthStar large‑model training framework developed by DeWu, detailing its background challenges, pipeline architecture, rich API support, multi‑GPU training modes, multi‑level embedding storage, hardware selection considerations, and a brief Q&A on data versus model parallelism.

AI FrameworkEmbedding Storagelarge model training

0 likes · 9 min read

Architect

Oct 18, 2024 · Operations

Design and Implementation of a Self‑Developed Video Transcoding Core Based on FFmpeg

This article describes the motivations, architecture, and key techniques of a custom video transcoding core built on FFmpeg, covering modular pipeline design, controllable serial/parallel execution, dynamic resolution and frame‑rate adaptation, SEI handling, and performance improvements for both live and on‑demand streaming.

Video Transcodingdynamic adaptationffmpeg

0 likes · 11 min read

Design and Implementation of a Self‑Developed Video Transcoding Core Based on FFmpeg

NetEase Yanxuan Technology Product Team

Jan 9, 2023 · Operations

Loggie: A High-Performance Log Collection Agent System Design and Implementation

Loggie is a cloud-native, Go-based log-collection agent that replaces Filebeat and Flume by using a micro-kernel producer-consumer architecture with hot-swappable pipelines, achieving 2 GB/s read speeds, 1.6‑2.6× higher throughput while using only a quarter of the CPU, and providing built-in observability, reliability, and latency monitoring for large-scale enterprise deployments.

GoOperationslog agent

0 likes · 16 min read

Loggie: A High-Performance Log Collection Agent System Design and Implementation

Bilibili Tech

Nov 8, 2022 · Industry Insights

BANG Engine: Multi‑Level Pipelines & GPU Acceleration for Faster Video Transcoding

To meet Bilibili’s demanding live and on‑demand video transcoding needs, the BANG engine combines a multi‑stage pipeline architecture, frame‑block and multi‑frame parallelism, SIMD‑based CPU acceleration, and TensorRT/TensorFlow GPU inference, offering configurable string‑based pipelines that dramatically increase throughput while simplifying integration.

BilibiliGPU AccelerationTensorRT

0 likes · 18 min read

BANG Engine: Multi‑Level Pipelines & GPU Acceleration for Faster Video Transcoding

Youku Technology

Nov 2, 2021 · Game Development

Building a High-Performance, High-Reusability, High-Reliability Audio Rendering Engine: Youku's Practice

Youku’s commercial‑grade audio rendering engine achieves high performance, reusability, and reliability by modularizing audio interfaces, post‑processing, output, caching, and focus management across multiple OSes, employing chain‑style pipelines, reactive filters, double‑linked buffer caching, latency monitoring, exception detection, and spatial‑audio filters for 5.1 surround sound.

C++ audio processingYoukuaudio rendering engine

0 likes · 11 min read

Building a High-Performance, High-Reusability, High-Reliability Audio Rendering Engine: Youku's Practice