Tagged articles

Attention optimization

6 articles · Page 1 of 1

Jul 19, 2026 · Artificial Intelligence

Asymmetric KV Decoupling and Head-Level Competition: Three Strategies for Efficient Long-Text Reasoning

The article analyzes the fundamental trade‑off between global competition in attention and efficiency when processing million‑token contexts, and presents three complementary techniques—AsymKV, Compression‑Aware Tuning (CAT), and Softmax Linear Attention (SLA)—that preserve or rebuild global competition while dramatically reducing memory and compute costs, achieving state‑of‑the‑art results on LongBench and various retrieval benchmarks.

Asymmetric KVAttention optimizationCompression-aware training

0 likes · 14 min read

Asymmetric KV Decoupling and Head-Level Competition: Three Strategies for Efficient Long-Text Reasoning

AI Tech Publishing

Jun 29, 2026 · Artificial Intelligence

Understanding the Full LLM Inference Pipeline: From Tokenization to Streaming Output

The article walks through every stage of LLM inference—from tokenization and embedding, through transformer layers, prefill (compute‑bound) and decode (memory‑bound) phases, KV‑cache management, attention redesign, quantization, and serving infrastructure—explaining how each step consumes time and resources and how to identify bottlenecks.

Attention optimizationKV cacheQuantization

0 likes · 14 min read

Understanding the Full LLM Inference Pipeline: From Tokenization to Streaming Output

Machine Learning Algorithms & Natural Language Processing

May 20, 2026 · Artificial Intelligence

How New LLM Architectures Like Gemma 4 and DeepSeek V4 Cut Long‑Context Costs

The article surveys recent open‑weight LLM releases—Gemma 4, Laguna XS.2, ZAYA1‑8B and DeepSeek V4—detailing how KV‑cache sharing, per‑layer embeddings, layer‑wise attention budgeting, compressed convolutional attention and manifold‑constrained hyper‑connections dramatically reduce memory and compute for ultra‑long contexts while preserving model quality.

Attention optimizationKV cacheLLM

0 likes · 25 min read

How New LLM Architectures Like Gemma 4 and DeepSeek V4 Cut Long‑Context Costs

AI Explorer

May 7, 2026 · Artificial Intelligence

Nvidia Endorses Open-Source “Light-Speed” Inference Engine for Coding Agents

The article examines how Nvidia’s open-source ‘light-speed’ inference engine tackles the token-bloat and compute bottlenecks of modern coding agents by redesigning attention and memory management, enabling order-of-magnitude speed gains without losing accuracy, and reshaping the AI-as-a-service ecosystem.

AI inferenceAttention optimizationNVIDIA

0 likes · 6 min read

Nvidia Endorses Open-Source “Light-Speed” Inference Engine for Coding Agents

Architect

Apr 25, 2026 · Artificial Intelligence

DeepSeek V4: 1M‑Token Context’s Impact on Model, Inference, Cache & Agents

The DeepSeek V4 technical report shows how a 1 million‑token context forces a redesign of attention, KV‑cache, optimizer, quantization and inference budgeting, turning long‑context capability from a costly showcase into a production‑ready feature for agents, search and Chinese professional tasks.

1M contextAgentic SearchAttention optimization

0 likes · 28 min read

DeepSeek V4: 1M‑Token Context’s Impact on Model, Inference, Cache & Agents

DaTaobao Tech

Sep 27, 2023 · Artificial Intelligence

FlashAttention-2: Efficient Attention Algorithm for Transformer Acceleration and AIGC Applications

FlashAttention‑2 is an IO‑aware exact attention algorithm that cuts GPU HBM traffic through tiling and recomputation, optimizes non‑matmul FLOPs, expands sequence‑parallelism and warp‑level work distribution, delivering up to 2× speedup over FlashAttention, near‑GEMM efficiency, and enabling longer‑context Transformer training and inference for AIGC with fastunet and negligible accuracy loss.

AIGCAttention optimizationDeep Learning

0 likes · 20 min read

FlashAttention-2: Efficient Attention Algorithm for Transformer Acceleration and AIGC Applications