Tagged articles
LLM optimization
26 articles · Page 1 of 1
This article details practical integration of Jev, a structured decision model, into three AI testing workflows—test case labeling, UI automation, and knowledge base retrieval—showing 10–93× latency reduction and 31–93× cost savings versus general LLMs, with code examples and architectural insights.
AI workflowsJevLLM optimization
0 likes · 21 min read

OpenAI launches a 28-day improvement sprint starting with GPT-6 models 50% faster, introduces textGrain invisible watermarks for AI text detection with 80-95% accuracy on 200-400 token texts but vulnerable to editing, and begins testing image ads in ChatGPT for 1.2B weekly users.
AI programmingAI watermarkingChatGPT ads
0 likes · 15 min read

Xiaomi details the dual-system architecture behind Super XiaoAI: a fast/slow routing layer that cuts model latency 35% via no-think mode and RAG-Planning, plus a full-duplex dialogue engine with three-level context compression (60–86% token reduction) and four-type interruption handling for natural in-car conversation.
Fast/Slow SystemFull-Duplex DialogueLLM optimization
0 likes · 10 min read

This article explains Anthropic's Prompt Caching mechanism including cache_control configuration, automatic vs explicit breakpoint caching, prefix hierarchy, 20-block lookback window, 4-breakpoint limit, TTL pricing (0.1x read, 1.25x/2x write), observability metrics, and Spring AI integration with code examples.
AI EngineeringAnthropicCache Control
0 likes · 11 min read

The whitepaper explains how the SKILL.md‑based agent‑skill framework solves four major LLM pain points—prompt bloat, missing procedural memory, costly multi‑agent ops, and cross‑vendor migration—by introducing a three‑stage progressive loading mechanism, rigorous evaluation standards, and meta‑skill automation for scalable, low‑token AI agents.
AGENTS.mdLLM optimizationMCP
0 likes · 35 min read

The article explains how AI agents use memory to retain conversation context, why sending the full history to large language models is inefficient, and presents rolling context windows, inverted‑index pruning, semantic embedding retrieval, and GraphRAG as complementary strategies to build more accurate and personalized agents.
AI MemoryGraphRAGInverted Index
0 likes · 10 min read

DeepSeek recently opened the Tile Kernels repository and released DeepEP V2, detailing new GPU kernel features, a fully JIT-enabled expert parallelism redesign that boosts peak performance by up to 1.3× while cutting SM usage fourfold, and hinting at an upcoming V4 release.
DeepEP V2DeepSeekExpert Parallelism
0 likes · 6 min read

The article presents five actionable strategies—defining a three‑dimensional performance baseline, applying layered injection load tests, co‑optimizing dynamic quantization with cache, employing SLO‑driven chaos engineering, and shifting testing left to compilation—to reliably measure and improve LLM throughput, latency, and resource efficiency in production.
LLM optimizationLarge Language ModelsQuantization
0 likes · 7 min read

This article explains the prompt‑caching mechanism used by Claude Code, showing how separating static prefixes from dynamic tails and leveraging KV‑tensor caching reduces the O(n²) complexity of transformer pre‑fill to O(n), achieving a 92% cache hit rate and up to 81% cost savings in long‑running AI agent sessions.
AI AgentsClaudeLLM optimization
0 likes · 12 min read

The article presents CharacterFlywheel, a 15‑generation flywheel methodology that iteratively improves social‑dialogue LLMs in production using data‑driven reward models, rejection sampling, and a mix of SFT, DPO, and RL, with detailed experiments and best‑practice insights.
AI safetyData PipelineLLM optimization
0 likes · 12 min read

This article reviews recent 2025 advances in long‑sequence LLM inference, covering Kimi Linear attention, DuoAttention and DeepSeek Sparse Attention, MegaKernel and MPK designs for kernel‑level efficiency, reinforcement‑learning rollout optimizations, and the Tawa deep‑learning compiler framework.
Deep Learning CompilerLLM optimizationLinear attention
0 likes · 22 min read

The paper introduces GLM, a multi‑agent Graph‑CoT framework with an optimized LLM serving architecture that dramatically improves accuracy, reduces token consumption, lowers latency, and increases throughput across diverse domains, as demonstrated by extensive GRBench evaluations.
LLM optimizationbenchmark evaluationgraph reasoning
0 likes · 10 min read

ktransformers is an open‑source AI model optimization framework that dramatically reduces memory usage and speeds up loading and inference, enabling ordinary laptops— even without a GPU— to run 7B‑13B large language models for coding, content creation, and academic assistance.
KTransformersLLM optimizationPython
0 likes · 10 min read

This article distills Anthropic’s “Effective Context Engineering for AI Agents” into key insights, explaining why context engineering matters, how it differs from prompt engineering, what constitutes good practice, and practical techniques—system prompts, tool design, few‑shot prompting, compaction, structured note‑taking, and sub‑agent architectures—to mitigate context overload in large language model agents.
AI AgentsLLM optimizationagent design
0 likes · 10 min read

This article explores the emerging concept of context engineering for AI agents, explains why managing long‑range context is critical, and details five practical strategies—Offload, Reduce, Retrieve, Isolate, and Cache—backed by insights from leading industry teams and the "Bitter Lesson" philosophy.
AI AgentsLLM optimizationcontext engineering
0 likes · 30 min read

In a two‑hour interview, DeepVista CEO Jing Conan Wang explains how his new "context modeling" paradigm addresses the rigidity, lack of personalization, and performance limits of current RAG‑based AI agents, proposing a dual‑model architecture that learns and adapts context dynamically for faster, more accurate results.
AI ArchitectureLLM optimizationPersonalized AI
0 likes · 15 min read

This article explains how Alibaba Cloud's AI platform PAI leverages the EasyDistill framework for post‑training model optimization, covering knowledge distillation concepts, data synthesis techniques, basic and advanced distillation training, the DistilQwen model family, real‑world customer cases, and step‑by‑step practical demos.
AI platformEasyDistillLLM optimization
0 likes · 12 min read

The article introduces SRPO, a two‑stage history‑resampling reinforcement‑learning framework that systematically tackles common GRPO training issues and achieves state‑of‑the‑art performance on both math and code benchmarks with far fewer training steps, while also revealing emergent self‑reflection behaviors in large language models.
LLM optimizationSRPOcross-domain training
0 likes · 12 min read


AIWalkerFeb 19, 2025 · Artificial Intelligence
DeepSeek introduces the NSA sparse attention mechanism, combining dynamic hierarchical sparsity, coarse token compression and fine token selection to achieve up to 11.6× faster inference, lower pre‑training cost, and superior benchmark performance across general, long‑context, and chain‑of‑thought tasks.
BenchmarkDeepSeekLLM optimization
0 likes · 9 min read

The article enumerates DeepSeek’s extensive technical optimizations—including Grouped Query Attention, Multi‑head Latent Attention, Mixture‑of‑Experts, 4D parallelism, quantization, and multi‑token prediction—that together enable cheap, high‑performance large language models.
4D parallelismDeepSeekGrouped-Query Attention
0 likes · 8 min read

This article presents a comprehensive analysis of Graph RAG, detailing its implementation workflow, step‑by‑step execution guide, four targeted optimization strategies, and experimental validation that demonstrates significant improvements in global and local question answering for industry scenarios.
Entity ExtractionGraph RAGLLM optimization
0 likes · 18 min read

This article provides a thorough, four‑part overview of RLHF for large language models, covering preference‑optimization algorithms (PPO‑based and offline RL approaches), reward‑model training techniques, inference‑time exploration strategies, and practical implementation details including the OpenRLHF framework and resource‑allocation tricks.
DPOLLM optimizationOpenRLHF
0 likes · 27 min read

This guide walks readers from beginner to proficient Prompt Engineer by outlining the evolution of prompting, introducing a universal four‑component template, and detailing a five‑step workflow—including refinement, retrieval‑augmented generation, chain‑of‑thought reasoning, and advanced tuning techniques—plus evaluation metrics for LLM performance.
AI promptingChain-of-ThoughtLLM optimization
0 likes · 51 min read

This article surveys the main challenges of deploying large language models, introduces key RAG optimization papers such as RAPTOR, Self‑RAG, and CRAG, and compiles practical engineering tricks—including chunking, query rewriting, hybrid and progressive retrieval—to help practitioners build more accurate and efficient RAG systems.
AI researchLLM optimizationRAG
0 likes · 22 min read

In the AI‑native era, Baidu's ad platform adopts a multi‑agent architecture that combines large and small LLMs, SOP‑driven workflows, long‑term memory, and vector databases to achieve high query accuracy, low latency, and significant business gains while tackling challenges such as hallucination, planning, execution, and personalization.
AI AgentsAdvertising PlatformLLM optimization
0 likes · 18 min read

Since 2021, NetEase Cloud Music’s Design Platform has evolved its design‑frontend workflow through three stages—engineering phase 1.0, phase 2.0, and the AI‑driven intelligent phase—by introducing the C2D2C (Code‑to‑Design‑to‑Code) methodology, unified design systems, LLM‑enhanced code, and generative AI tools, cutting communication overhead and boosting designer and developer productivity by up to 200 %.
AI designC2D2CD2C
0 likes · 31 min read
