Tagged articles
latency optimization
16 articles · Page 1 of 1
This article categorizes eight technical approaches to full-duplex speech systems — from cascaded pipelines and decision-ear classifiers to native multi-stream models and expressive TTS — using an academic framework that distinguishes true full-duplex (simultaneous listen/speak, overlap, interruption) from pseudo full-duplex (millisecond state-machine switching). It benchmarks latency, data requirements, auditability, and deployment fit, citing key papers and open-source implementations.
benchmarkingcascaded pipelinedecision ear
0 likes · 21 min read

OpenAI's Pando Git backend separates read and write paths, stores packs immutably in object storage, and broadcasts only the main branch, reducing median push latency to one-fifth while serving 1 PB of Git data daily and 57 TB/hour with 99% cache hits.
GitObject StorageOpenAI
0 likes · 5 min read

This article dissects Perplexity's production system prompt for its search agent, revealing how a concise 1,000-word prompt with eight tools enforces tool-first behavior, parallel calls, a three-call limit, and direct-answer formatting to eliminate hallucination and latency.
AI agentsPerplexitylatency optimization
0 likes · 12 min read

The article benchmarks Alibaba Cloud AI Gateway Serverless Enterprise Edition against self-hosted Higress, showing up to 97.1% lower gateway processing latency for 1MB requests and 90.2% for 10MB requests at 10-100 concurrency, attributing gains to native Envoy plugin optimizations reducing memory copies and Wasm linear memory limitations, while also comparing operational costs and maintenance trade-offs.
AI gatewayEnvoyHigress
0 likes · 10 min read

Within a week of TypeSafe's Jev release, 13 arXiv papers benchmark the decision model across edge orchestration, agent memory, judging, scam detection, video quality, and visual tasks, showing 15-26% lower latency and 70% cost reduction versus LLMs, but with accuracy gaps on complex reasoning and sensitivity to option naming.
Agent MemoryDecision ModelsJev
0 likes · 15 min read

Chengdu Guanghangxin's fully domestic 24-port 16Gbps FC SAN switch achieves sub-350ns latency, surpassing Broadcom's 580ns, leveraging lossless BB Credit flow control and crossbar architecture; the article details FC technology, market trends, and domestic breakthroughs in a market dominated by Brocade and Cisco.
BB CreditFC SANFibre Channel
0 likes · 18 min read

This article explains a voice agent architecture that separates real-time full-duplex frontend interaction from asynchronous backend computation using delegation IDs, detailing the two-timeline mechanism, three-layer responsibility split, task state machine, and evaluation strategies to avoid stale responses and interruption mishandling.
async backenddelegation patternfull-duplex
0 likes · 8 min read

The article explains that LLM prompt caching reuses internal KV states rather than full answers, compares provider implementations, quantifies cost and latency savings, and provides concrete guidelines for structuring prompts to maximize cache hits, along with monitoring signals and a practical evaluation checklist.
AI inferenceLLMPrompt Caching
0 likes · 13 min read

The article examines common pitfalls of naïve ad insertion in short‑video feeds, explains how cursor‑based pagination prevents duplicate ads, and outlines a client‑side mixing architecture that isolates services, meets strict latency requirements, and ensures accurate ad billing.
Cursor Paginationad mixingbackend design
0 likes · 4 min read

The paper introduces LEADRE, a multi‑faceted knowledge‑enhanced large language model‑driven display advertisement recommender that tackles user interest modeling, knowledge alignment, and low‑latency deployment, achieving significant GMV gains in Tencent’s ad platforms through innovative prompt engineering, semantic alignment, and TensorRT‑accelerated inference.
Knowledge AlignmentLLMTensorRT
0 likes · 16 min read

Bilibili's R&D built a cloud-based broadcast console for the 2024 CCTV Spring Festival Gala, delivering 4K HDR streaming, AI SDR-to-HDR conversion, low latency, bandwidth‑aware transcoding, and a synchronized “send bullet screen” interactive feature using custom SEI timestamps for hundreds of millions of viewers.
HDRSEIcloud broadcasting
0 likes · 15 min read

Async‑fork shifts the costly page‑table copying from Redis’s parent process to its child, allowing the parent to resume handling queries instantly and cutting snapshot‑induced latency spikes by over 98%, thereby dramatically improving tail latency during AOF rewrites, RDB backups, and master‑slave synchronizations.
Async-forkPage TableRedis
0 likes · 21 min read

This comprehensive guide explains Linux kernel performance profiling—both on‑CPU and off‑CPU—by stressing the need to target the critical 3 % of code, covering throughput, latency and power metrics, scalability laws, flame‑graph visualizations, perf and eBPF tools, lock‑contention analysis, and further reading recommendations.
Flame GraphLinux kerneleBPF
0 likes · 27 min read

This talk recounts how the Good Doctor platform tackled severe online incidents by launching the DOA project, then a service risk governance initiative that identifies, quantifies, and mitigates latency‑related risks through metrics‑driven development, dependency analysis, middleware reliability, and a dedicated risk‑management platform.
SRElatency optimizationmicroservices
0 likes · 16 min read

By replacing the default Parallel GC with a ParNew‑CMS collector, enlarging the Young generation, fixing Metaspace settings, and tuning CMS occupancy thresholds, the video service cut Young and Full GC pauses dramatically, lowered Full GC count by over 80%, and achieved more than 30% P99 latency reduction, with some APIs improving up to 80%.
CMSJVMJava
0 likes · 16 min read

By adding model warm‑up files, separating load/unload threads, switching to the Jemalloc allocator, and isolating TensorFlow’s parameter memory from RPC request buffers, iQIYI’s engineers reduced TensorFlow Serving hot‑update latency spikes in high‑throughput CTR recommendation services from over 120 ms to about 2 ms, eliminating jitter.
Model Hot UpdateTensorFlow Servingjemalloc
0 likes · 11 min read
