Tagged articles

Long-context LLM

4 articles · Page 1 of 1

Jul 19, 2026 · Artificial Intelligence

Asymmetric KV Decoupling and Head-Level Competition: Three Strategies for Efficient Long-Text Reasoning

The article analyzes the fundamental trade‑off between global competition in attention and efficiency when processing million‑token contexts, and presents three complementary techniques—AsymKV, Compression‑Aware Tuning (CAT), and Softmax Linear Attention (SLA)—that preserve or rebuild global competition while dramatically reducing memory and compute costs, achieving state‑of‑the‑art results on LongBench and various retrieval benchmarks.

Asymmetric KVAttention optimizationCompression-aware training

0 likes · 14 min read

Asymmetric KV Decoupling and Head-Level Competition: Three Strategies for Efficient Long-Text Reasoning

Data Party THU

May 16, 2026 · Artificial Intelligence

SubQ Beats Transformers: 12‑Million‑Token Context Model at Only 5% of Opus Cost

The article analyzes SubQ, a new LLM architecture using Subquadratic Sparse Attention (SSA) to achieve a 12‑million‑token context window with linear compute scaling, delivering up to 52× speedup and costing just 5% of Opus while matching dense‑attention performance on long‑context benchmarks.

Long-context LLMSSASubQ

0 likes · 14 min read

SubQ Beats Transformers: 12‑Million‑Token Context Model at Only 5% of Opus Cost

Machine Learning Algorithms & Natural Language Processing

Mar 7, 2026 · Artificial Intelligence

How Princeton’s DYSCO Decoder Boosts Long-Context Reasoning by 25% Without Fine‑Tuning

The DYSCO (Dynamic Attention‑Scaling Decoding) algorithm, introduced by Princeton’s Chen Danqi team and NYU, eliminates the need for fine‑tuning and restores performance on long‑context tasks, delivering up to a 25% relative gain on 128K token benchmarks while adding only about 3.8% extra FLOPs.

DYSCODynamic Attention ScalingInference Optimization

0 likes · 10 min read

How Princeton’s DYSCO Decoder Boosts Long-Context Reasoning by 25% Without Fine‑Tuning

NewBeeNLP

Apr 11, 2024 · Artificial Intelligence

How BAMBOO Benchmarks Long-Context LLMs: Design, Tasks, and Key Findings

The article introduces the BAMBOO benchmark for evaluating large language models on long-text tasks, outlines its four design principles, describes ten datasets across five tasks, presents experimental results on five models, and discusses five research questions and future directions for improving long-context modeling.

Artificial IntelligenceLong-context LLM

0 likes · 9 min read

How BAMBOO Benchmarks Long-Context LLMs: Design, Tasks, and Key Findings