BestHub
Sign in
Tag

AI systems

0 views collected around this technical thread.

Architect
Architect
Jul 2, 2024 · Artificial Intelligence

Mooncake: A Separated Architecture for Large‑Language‑Model Inference

The article presents Mooncake, a split‑architecture inference platform for the Kimi LLM assistant, detailing its three elastic resource pools, the rationale for using Time‑Between‑Tokens over TPOT, and design choices for Prefill, KVCache, and Decode stages to improve latency and throughput.

AI systemsKVCacheLLM inference
0 likes · 9 min read
Mooncake: A Separated Architecture for Large‑Language‑Model Inference
DataFunTalk
DataFunTalk
Nov 14, 2019 · Artificial Intelligence

Building the Most Reliable Autonomous Driving Infrastructure at Pony.ai

This article outlines Pony.ai's comprehensive autonomous driving infrastructure, describing traditional internet back‑end components, additional vehicle‑mounted systems, large‑scale simulation, data challenges, and the reliability, performance, and flexibility practices needed to support rapid growth and safe robotaxi operations.

AI systemsData EngineeringInfrastructure
0 likes · 15 min read
Building the Most Reliable Autonomous Driving Infrastructure at Pony.ai