Tagged articles
long context
76 articles · Page 1 of 1
Prefrontal AI and ETH Zurich introduce Latent Sparse Attention (LSA), which clusters keys in a learned latent space and uses hierarchical bucket search to reduce indexer complexity from quadratic to O(T log T), achieving 22.7x prefill speedup over GLM-5.3-Flash and 72.4x over Qwen3.8-27B at 10M tokens while maintaining accuracy.
DeepSeekGLM-5.3-FlashLLM inference
0 likes · 18 min read

A hands-on review of Claude Fable 5.2 reveals improved aesthetics, 1M token context, 20% price drop, 30% speed boost, and the ability to orchestrate 1,000+ sub-agents over 8 days to build a prototype OS, plus one-prompt video and 3D scene generation.
3D scenesAI model evaluationClaude Fable 5.2
0 likes · 14 min read

Alibaba's Qoder platform quietly added an anonymous model called Qoduck, free for personal users for a limited time, speculated to be a Qwen 4 Max preview, offering strong cross-file refactoring, long-context understanding, and agent-like planning with no usage credits required during the trial.
AI coding assistantAlibabaQoder
0 likes · 5 min read

A Japanese developer implemented Qwen Sparse Attention (QSA) on Qwen3.8-27B, achieving 2.92x faster prefill and 1.79x faster decode at 261K tokens while maintaining perplexity and DocVQA scores; a quantized version further reaches 2375 tok/s prefill and 115 tok/s decode with 22.8 GiB VRAM.
Community DevelopmentInference OptimizationQSA
0 likes · 5 min read

Baidu releases ERNIE 5.5 Flash Base, a 280B sparse MoE model with 14B activated parameters, 128K training context extended to 1M at inference, featuring hybrid sparse attention (MLA+HCA), multi-router MoE, and ECHO future-token prediction, achieving SOTA benchmarks against DeepSeek-V4-Flash and other peers.
BenchmarkECHOERNIE 5.5 Flash Base
0 likes · 18 min read

The Strata inference engine enables Qwen3.8-Flash-Next (180B total params, 6B active) to achieve 3145 tokens/sec prefill and 120 tokens/sec decode at 400K context on dual RTX 3090s by offloading experts to CPU/SSD, using speculative decoding and chunked prefill, offering a low-cost alternative to Mac Studio for long-context workloads.
LLM inferenceQwen3.8-Flash-NextRTX 3090
0 likes · 11 min read

This article traces the evolution of attention mechanisms from early visual attention and RNN-based attention through the Transformer revolution to modern sparse, linear, and state-space variants, detailing recent innovations like GQA, MLA, dynamic sparse attention (MoBA, NSA), hybrid architectures, and cross-layer memory sharing in open-source LLMs such as Llama, DeepSeek, and Qwen.
Attention MechanismCross-Layer SharingDynamic Sparse Attention
0 likes · 17 min read

IQuest Research releases technical report for IQuest-Q1, a 320B MoE model with 15B active parameters, 512K context window, recursive multi-token prediction, and agent-focused post-training, achieving strong benchmarks across coding, tool use, and reasoning tasks.
Agent TrainingMixture of ExpertsRecursive MTP
0 likes · 16 min read

STILL introduces a content-aware intra-layer hybrid attention mechanism that routes tokens to Softmax or linear attention based on self-saliency scores computed locally, preserving pre-trained norm structure via NP-Map, achieving 80% memory reduction and 83% prefill speedup with only 0.04B training tokens.
Efficient TransformersIntra-Layer Hybrid AttentionLLM Linearization
0 likes · 20 min read

TypeSafe releases Jev 1.14.0 with multimodal image support, a new box output type for object localization, and a 262K token context window, all gated behind explicit danger-zone headers to signal experimental status and encourage careful testing before production deployment.
AI model releaseJev 1.14.0TypeSafe
0 likes · 8 min read

Alibaba Cloud's Zhenwu M890 supernode (64 chips, 800GB/s interconnect, 9TB unified memory) adapted StepFun's Step 5 Preview (600B sparse MoE, 27B active, 1M context, multimodal) with operator fusion, expert parallel optimization, and prefill-decode separation, achieving lower TTFT, higher decode throughput, and stable 1M context support.
Inference OptimizationMoEStep 5 Preview
0 likes · 6 min read

This article chronicles DeepSeek's paper timeline from 2023 to 2026, analyzing how each model generation optimized training compute, KV cache, and throughput via MoE, MLA, sparse attention, and conditional memory, shifting long-context competition from maximum length to per-token byte cost.
DeepSeekGRPOKV cache optimization
0 likes · 37 min read

The author tests Alibaba's Qoder AI coding tool with the newly free Qwen3.8-Flash model, using it to rebuild a complex audio studio app with 14 requirements, demonstrating SubAgent parallelism, site templates, and long-context handling, concluding the free model suffices for most development tasks.
AI coding assistantAI-assisted developmentAlibaba
0 likes · 9 min read

This analysis of GPT-6 Astra explains its strength in chaining reasoning, coding, browsing, and documentation into continuous workflows, advises using its long context with clear goals and constraints, and recommends reserving it for complex multi-step tasks like cross-file refactoring rather than simple code generation.
AI-assisted codingDeveloper ProductivityGPT-6 Astra
0 likes · 3 min read

This article analyzes GPT-6 Astra's strengths in long-context multi-step tasks like codebase analysis and browser automation, outlines its limitations regarding input quality, tool permissions, and media support, and advises developers to define clear goals and verification steps for reliable results.
AI-assisted programmingGPT-6 Astracode generation
0 likes · 4 min read


ArchitectSep 14, 2026 · Artificial Intelligence
This article dissects DeepSeek V4.1 Flash's architecture for 1M-token context, explaining how CED reduces prefill compute, CSA2 shares global KV across layers, hierarchical sparse indexing narrows search, FP4 quantization shrinks storage, SWA uses bounded replay for recovery, and Engram adds a conditional memory path—revealing five cost dimensions of long-context deployment.
Bounded ReplayCEDCSA2
0 likes · 21 min read

Deep dive into Moonshot AI's Kimi Work desktop agent: 24/7 Goal-mode execution, browser automation via logged-in sessions, 300-agent parallel swarms, native financial/academic data integration, and hard limits — 62 tokens/sec, 51% hallucination rate, 50-round memory decay, and a 48-hour compute crunch that halted new subscriptions.
AI AgentsCompute ConstraintsGoal mode
0 likes · 14 min read

Microsoft research shows sliding window attention with four attention sinks achieves 99% performance recovery on knowledge tasks without any post-training, matching linear attention methods requiring hundreds of millions of tokens while outperforming them 2-10x on long-context benchmarks.
KV CacheLinear attentionMicrosoft Research
0 likes · 9 min read

GLM‑5.3 introduces a 1 million‑token context window, 128 K output limit, mandatory reasoning, and new OpenAI‑compatible protocols, delivering up to 50 % better programming performance, stronger security‑vulnerability detection, and detailed migration guidance for developers in production.
AI codingBenchmarkGLM-5.3
0 likes · 15 min read

The author downloads the 52 GB Qwen3.8‑27B model, runs it natively in BF16 on a 96 GB RTX PRO 6000 without quantization, benchmarks speculative decoding (MTP) and long‑context performance up to 262 K tokens, and then connects the model to DeepSeek Harness to operate as a coding agent.
Coding AgentDeepSeek HarnessQwen3.8-27B
0 likes · 12 min read

MIT CSAIL’s study shows that training a Recursive Language Model with a harness that keeps each model call locally in-distribution allows the system to extrapolate up to 32-fold longer sequences, achieve superior cross-domain transfer, and outperform transformer baselines despite higher training cost.
GeneralizationHarnessRLM
0 likes · 10 min read

HiLS-Attention introduces a hierarchical landmark sparse attention that mathematically resolves chunk‑importance estimation and end‑to‑end differentiability, matching full‑attention perplexity while delivering 13.5× faster prefill, 15.7× faster decode, and up to 512‑fold context extrapolation, even surpassing full attention on long‑context retrieval tasks.
EfficiencyHiLS-AttentionTransformer
0 likes · 11 min read

The live discussion reveals that stronger models hide subtle errors, shifting from chatbots to agents requires a cognitive upgrade, multi‑agent collaboration hinges on clear contracts, physical permissions trump prompts, and a three‑layer Rule‑Skill‑Hook framework plus careful handling of long context and self‑evolution are essential for reliable, cost‑effective enterprise AI deployment.
AI AgentsAgent OrchestrationConstraint engineering
0 likes · 17 min read

The author walks through a complete Qwen3.7-Max workflow—knowledge collection, cross‑document understanding, article creation, and video production—highlighting tips, challenges, and the model's ability to handle 1 M‑token contexts and long‑chain tasks.
AI workflowArticle GenerationKnowledge Extraction
0 likes · 10 min read

A joint effort by Baidu Baige and Fudan University introduces the LU‑KV framework, which treats KV‑cache budget allocation as a global combinatorial optimization problem, achieving only 0.52% relative performance loss at 80% compression and establishing a new efficiency‑accuracy SOTA on LongBench.
Cache EvictionEfficiencyICML 2026
0 likes · 5 min read

Researchers from CMU and collaborators propose a ‘sleep’ phase for transformer‑based LLMs that repeatedly re‑processes accumulated context to update fast weights in a state‑space module, enabling memory consolidation that reduces KV‑cache pressure and markedly improves performance on long‑context, multi‑step reasoning benchmarks.
LLMSSMfast weights
0 likes · 10 min read

The article explains why Retrieval‑Augmented Generation (RAG) addresses the static knowledge limitation of large models, contrasts its role of “what to say” with fine‑tuning’s focus on “how to say,” compares costs and performance against long‑context models, and offers a practical hierarchy (Prompt → RAG → LoRA/QLoRA fine‑tuning → Distillation) plus best‑practice combinations.
AI AgentsLLMRAG
0 likes · 9 min read

The article breaks down eight technical terms—frontier coding, 1M‑long context, native multimodal, open‑source levels, benchmark layers, CUDA operators, autonomous iteration, and verifiable engineering strength—to help readers understand what modern AI model release notes actually mean.
BenchmarkCUDA operatorsModel Evaluation
0 likes · 11 min read

The article presents a technical deep‑dive into TencentDB Agent Memory’s short‑term memory compression, which combines context offloading and a Mermaid‑based infinite canvas to reduce token usage by up to 61 % while improving task success rates by over 50 % across multiple long‑session benchmarks.
AgentLLMMemory Compression
0 likes · 45 min read

Recent open‑weight LLMs such as Gemma 4, Laguna XS.2, ZAYA1‑8B, and DeepSeek V4 introduce KV‑cache sharing, per‑layer embeddings, layer‑wise attention budgeting, and compressed attention mechanisms that dramatically reduce memory and compute overhead for very long contexts while preserving model quality.
KV sharingLLMarchitecture
0 likes · 25 min read


Wuming AIMay 10, 2026 · Artificial Intelligence
The article examines why a model’s advertised context window (e.g., 128 K or 1 M tokens) does not guarantee effective long‑context reasoning, summarizing the RULER framework that breaks long‑context ability into retrieval, interference resistance, multi‑hop tracking, aggregation, and multi‑answer recall, and offering practical guidance for evaluating and using such models.
LLMRULERaggregation
0 likes · 16 min read

The SubQ model introduces Subquadratic Sparse Attention (SSA), a content‑dependent routing mechanism that reduces attention complexity to linear, enabling a 12‑million‑token context window with a 52.2× speedup and only 5% of Opus's cost, as demonstrated on MRCR v2, RULER, and SWE‑Bench benchmarks.
LLMSubQlong context
0 likes · 14 min read

The article analyzes how expanding LLM context windows to millions of tokens reshape Retrieval‑Augmented Generation, detailing chunking trade‑offs, embedding retrieval limits, attention U‑shaped distribution, benchmark results, and the emerging practice of Context Engineering for optimal end‑to‑end pipelines.
LLMRAGbenchmarking
0 likes · 10 min read

The Latent‑Condensed Attention (LCA) method dramatically cuts KV‑cache memory by 90%, speeds up pre‑fill by 2.5× and reduces decode latency by 1.8× for 128K‑token contexts, while requiring no extra parameters and preserving model performance across diverse LLMs.
KV cache reductionLCATransformer
0 likes · 10 min read

DeepSeek V4 introduces a suite of architectural innovations—including mixed attention, manifold‑constrained hyper‑connections, the Muon optimizer, and FP4‑aware quantization—that together slash million‑token inference cost to a tenth of its predecessor while delivering benchmark results that rival top‑tier closed‑source models.
BenchmarkEfficiencyFP4 quantization
0 likes · 44 min read


ArcThinkApr 27, 2026 · Artificial Intelligence
GPT‑5.5, the first fully retrained base model since GPT‑4.5, delivers an 11.7‑point jump on ARC‑AGI‑2, wins 9 of 10 shared benchmarks, shows superior agent and ultra‑long‑context performance, yet incurs higher latency and token pricing, while Claude Opus 4.7 excels on deep‑reasoning tasks, marking a multi‑pole era for frontier AI.
AI benchmarksAgent CapabilitiesClaude Opus 4.7
0 likes · 16 min read


ArcThinkApr 27, 2026 · Artificial Intelligence
GPT‑5.5, the first fully retrained base model since GPT‑4.5, delivers an 11.7‑point jump on ARC‑AGI‑2, dramatic long‑context gains, and wins 9 of 10 shared benchmarks against GPT‑5.4, while a side‑by‑side comparison with Claude Opus 4.7 shows each model excelling in different domains, heralding a multi‑polar era for frontier AI.
AgentBenchmarkClaude Opus 4.7
0 likes · 16 min read


CodeTrendApr 26, 2026 · Artificial Intelligence
DeepSeek V4, released in April 2026, introduces two versions—Pro and Flash—with up to 1.6 trillion parameters and a million‑token context window, leveraging hybrid attention, compressed KV cache, and specialized training techniques to dramatically cut hardware dependence and inference cost.
DeepSeekFP4Mixture of Experts
0 likes · 5 min read

DeepSeek released the V4 series—V4‑Pro (1.6 T total, 49 B active) and V4‑Flash (284 B total, 13 B active)—featuring three architectural upgrades, three inference modes, mixed‑precision FP4/FP8 weights, and benchmark results that place its programming ability at the top of open‑source models while supporting a million‑token context window.
AI ArchitectureBenchmarkDeepSeek
0 likes · 5 min read

DeepSeek V4, released shortly after GPT‑5.5, offers two models—V4‑Pro (1.6 T parameters) and V4‑Flash (284 B parameters)—that introduce a hybrid CSA/HCA attention architecture to enable efficient million‑token context, achieve dramatic FLOPs and KV savings, deliver competitive programming and agent benchmarks, and adopt a disruptive pricing strategy, while also exposing training‑stability tricks and highlighting both strengths and remaining gaps.
BenchmarkLLMTraining Stability
0 likes · 25 min read

Two heavyweight LLMs launched on the same day—DeepSeek V4 emphasizing open, ultra‑long‑context, deployable foundations, and GPT‑5.5 pushing agentic, tool‑using execution—highlight a clear industry fork between owning work context and delegating task execution.
Agentic AIDeepSeekGPT-5.5
0 likes · 13 min read

DeepSeek V4 launches two models—Flash and Pro—both supporting up to 1 million token context and 384 K output tokens, offering non‑thinking and thinking modes with a reasoning_effort parameter, and featuring mixed attention, manifold‑constrained hyperconnections, a Muon optimizer, massive training data, and up to 73% FLOPs reduction versus V3.
AI ModelCambriconHuawei Ascend
0 likes · 5 min read

Claude Opus 4.7, announced as Anthropic’s most capable publicly available model, dramatically improves visual reasoning, long‑context task handling and instruction following, delivering up to a 2.4‑fold boost on benchmarks such as XBOW, SWE‑bench and structural biology, while also introducing new security guardrails and token‑usage costs.
AI benchmarksAnthropicClaude Opus 4.7
0 likes · 11 min read

Google Research introduces Memory Caching (MC), a technique that gives RNNs growing memory capacity, bridging the gap with Transformers to enable ultra‑long context processing while reducing memory demands, and demonstrates its effectiveness through extensive language‑modeling and recall experiments.
AI ArchitectureGoogle ResearchMemory Caching
0 likes · 7 min read

The open‑source GLM‑5.1 model, adapted to Baidu Baige's Kunlun XPU via the vLLM‑Kunlun Plugin, delivers record‑breaking SWE‑bench scores, eight‑hour autonomous coding, long‑context handling up to 64K tokens, and scalable deployment across tens of thousands of chips, showcasing end‑to‑end AI acceleration.
GLM-5.1Kunlun XPUlong context
0 likes · 8 min read

InfLLM‑V2 introduces a dense‑sparse switchable attention framework that preserves the original dense‑attention parameters while enabling efficient long‑context training, matching full‑attention performance on benchmarks such as RULER, LongBench, and chain‑reasoning tasks, and delivering up to 2.3× end‑to‑end inference speedup without degrading short‑sequence abilities.
EfficiencyInfLLM-V2Transformer
0 likes · 16 min read


SuanNiFeb 26, 2026 · Artificial Intelligence
Alibaba’s newly released Qwen3.5 series—spanning 27B, 35B, and 122B parameter models—demonstrates how hybrid compute, high‑quality data, and reinforcement‑learning can boost multimodal understanding, ultra‑long‑context handling, and multilingual support while drastically lowering hardware requirements, marking a shift from pure scaling to efficient AI evolution.
AI Architecturelong contextmultilingual
0 likes · 7 min read

MiniCPM‑SALA introduces a hybrid sparse‑linear attention architecture that reduces quadratic compute and memory costs, achieves state‑of‑the‑art performance on long‑context benchmarks, and delivers up to 3.5× faster inference than full‑attention models on sequences up to 1 million tokens.
LLMLinear attentionlong context
0 likes · 17 min read

Community testing of DeepSeek's rumored V4‑Lite‑285B model using the OpenAI MRCR 8‑pin standard shows perfect 1.0000 scores on several 128K‑token samples and a 256K‑token sample, achieving 100% recall in native 256K context while longer contexts drop to about 60%, with a note that the "needle‑in‑a‑haystack" method may be exploitable by DSA mechanisms.
DeepSeekLLMlong context
0 likes · 3 min read

The article presents SALA, a sparse‑linear hybrid attention architecture that replaces full attention in 9B‑parameter models, achieving comparable accuracy while cutting compute and memory costs, enabling million‑token inference on a single RTX 5090 and delivering up to 3.5× speed‑up over Qwen3‑8B.
Hybrid Position EncodingLLM efficiencyLinear attention
0 likes · 18 min read

This article analyzes Sakana AI's three recent papers that challenge traditional Transformer long‑sequence handling by removing positional embeddings, reconstructing position awareness, and adding a fast‑weight external memory, showing how each approach improves ultra‑long text understanding.
Memory MechanismPositional EmbeddingTransformer
0 likes · 12 min read

DeepSeek’s upcoming V4 model, built on the newly released mHC (Manifold-Constrained Hyper-Connections) paper, demonstrates mathematically grounded training stability, 2%+ reasoning gains, and four‑fold residual bandwidth that enables ultra‑long code context, positioning it as a potentially game‑changing holiday gift for programmers.
AI ModelTraining Stabilitydeepseek-v4
0 likes · 8 min read

Recursive Language Models (RLM) offer a cost‑effective alternative to expanding LLM context windows by storing prompts as variables and enabling recursive calls, allowing models to process over 100,000 tokens, with experiments showing superior performance and lower median costs compared to baseline approaches.
AI researchLLM scalingRecursive Language Models
0 likes · 5 min read

The article analyzes the Tensor Product Attention (TPA) method presented at NeurIPS 2025, explaining how it factorizes Q, K, V tensors to drastically reduce KV cache size and attention complexity, and demonstrates superior convergence, lower perplexity, and faster inference on long‑sequence tasks compared with existing attention variants.
KV CacheRoPETensor Product Attention
0 likes · 11 min read

Claude Sonnet 4.5 is now available on Amazon Bedrock, offering upgraded agent capabilities such as tool use, memory management, and extended context handling, with new API features like smart window management, automatic tool‑record cleanup, and cross‑dialog memory, demonstrated through a Python Boto3 example that generates a detailed digital‑transformation plan.
AI AgentsAmazon BedrockBoto3
0 likes · 9 min read

Nvidia's Rubin CPX GPU, unveiled in September 2025, uses GDDR7 memory and a split‑stage architecture to dramatically boost token‑per‑second rates for long‑context inference, while its integration into third‑generation Oberon servers promises higher power density, improved ROI, and scalable data‑center deployments.
AI inferenceGPU architectureNVIDIA
0 likes · 9 min read

REFRAG (REpresentation For RAG) introduces a novel decoding framework that compresses, senses, and expands context using precomputed chunk embeddings, achieving up to 30.85× faster first-token generation and 16× larger context windows without sacrificing perplexity, as validated across diverse long‑context tasks.
LLMRAGReinforcement Learning
0 likes · 18 min read

DeepSeek V3.1, a 685‑billion‑parameter open‑source model, supports up to 128,000 tokens, delivers mixed‑architecture capabilities, matches top‑tier closed systems in benchmarks, and its rapid community adoption signals a shift toward democratized AI development and new industry dynamics.
AI performanceDeepSeeklarge language model
0 likes · 6 min read

The paper "Efficient Long Context Fine-tuning with Chunk Flow" introduces ChunkFlow, a training framework that reorganizes variable‑length sequences into fixed‑size chunks, achieving up to 4.53× speedup and more stable GPU memory usage for large language models.
ChunkFlowGPU optimizationLLM training
0 likes · 7 min read

MiniMax’s latest M1 model, unveiled after a $300 million funding round, showcases a 4.56‑trillion‑parameter hybrid‑expert architecture with lightning attention, supporting up to one million tokens, and leverages reinforcement‑learning techniques to enhance long‑context handling, inference efficiency, and system‑2 reasoning capabilities.
AI scalingReinforcement Learninghybrid attention
0 likes · 16 min read

iQIYI’s AI research team secured three paper acceptances—two at ACL 2025 (including a main conference and a Findings paper) and one at INTERSPEECH 2025—covering long‑context large language model evaluation, Chinese novel summarization, and efficient Thai speech recognition, with links to each work.
ACL 2025AI researchINTERSPEECH 2025
0 likes · 7 min read


AIWalkerJun 18, 2025 · Artificial Intelligence
Large language models are booming, and this article highlights six cutting‑edge research directions—LLM‑plus synthetic data, reward modeling, inference techniques, LLM‑as‑a‑Judge, safety alignment, and long‑context handling—each illustrated with recent papers, experimental results, and links to code repositories.
LLMReward Modelinginference
0 likes · 9 min read

In a deep dive into the Cursor team's podcast, core members dissect the current hurdles of AI programming agents, covering feedback‑mechanism design, reinforcement‑learning reward sparsity, tool‑chain integration, long‑context handling, and emerging attention mechanisms that shape the future of code‑centric AI.
AI programmingattention mechanismslong context
0 likes · 35 min read


AntTechApr 10, 2025 · Artificial Intelligence
At the ICLR 2025 live session in Singapore, Ant Group showcased four cutting‑edge papers—CodePlan, Animate‑X, Group Position Embedding, and OmniKV—demonstrating advances in large‑language‑model reasoning, universal character animation, layout‑aware document understanding, and efficient long‑context inference.
AI researchDocument Understandinglarge language models
0 likes · 6 min read


21CTOApr 7, 2025 · Artificial Intelligence
Meta's Llama 4 series introduces the Scout, Maverick, and Behemoth models—featuring Mixture‑of‑Experts architectures, unprecedented 10‑million‑token context windows, and state‑of‑the‑art performance across vision, language, and multimodal benchmarks—while emphasizing efficient training, open‑source availability, and robust safety safeguards.
AI safetyLlama 4Mixture of Experts
0 likes · 14 min read

Meta announced the Llama 4 series—Scout, Maverick and Behemoth—featuring multimodal capabilities, Mixture‑of‑Experts design, up to 10 million‑token context windows, and state‑of‑the‑art performance on STEM, multilingual and image benchmarks, with models now downloadable from llama.com and Hugging Face.
Llama 4Mixture of ExpertsModel Training
0 likes · 14 min read

The article analyses Google’s Gemini 2.5 Pro as a decisive shift toward a “Reasoning Model”, detailing its architectural focus on inference, benchmark breakthroughs such as Humanity’s Last Exam and GPQA Diamond, long‑context capability, multimodal strengths, Vibe‑coding experience, and the roadmap for future Gemini models.
AI strategyBenchmarkGemini 2.5 Pro
0 likes · 25 min read


ArchitectFeb 24, 2025 · Artificial Intelligence
The article details the development, architectural evolution, and practical challenges of MoBA—a sparse attention framework inspired by Mixture‑of‑Experts that scales LLM context length to 10 M tokens, supports seamless switching between full and sparse attention, and is now released as a minimal open‑source solution.
AI ArchitectureContext ParallelLLM training
0 likes · 13 min read

The article introduces MoBA, a Mixture‑of‑Block‑Attention mechanism that applies Mixture‑of‑Experts principles to transformer attention, enabling efficient long‑context processing for large language models while maintaining performance comparable to full attention through sparse, trainable block selection and seamless switching.
Attention MechanismLLMMixture of Experts
0 likes · 12 min read

Index-1.9B-32K is a 1.9B-parameter model with a 32K token context window, achieving strong long‑text performance comparable to larger models while using only about 2% of GPT‑4’s compute, trained via long pre‑training and supervised fine‑tuning, with a trade‑off of reduced short‑context ability.
AIPretrainingevaluation
0 likes · 12 min read


NewBeeNLPAug 3, 2024 · Artificial Intelligence
This article reviews recent research on extending large language model context windows to millions of tokens, covering SAMBA's hybrid architecture, Contextual Position Encoding (CoPE), RoPE base length theory, Retrieval Head analysis, and the memory‑efficient Infini‑Attention mechanism.
LLM researchefficient attentionlarge language models
0 likes · 10 min read

In a May 15 round‑table, experts debated the validity of scaling laws, the role of synthetic and semi‑synthetic data in overcoming data bottlenecks, explored alternatives to the Transformer such as RNN‑based and hybrid designs, evaluated the practicality of Mixture‑of‑Experts models, and examined two main strategies—KV‑cache compression and input‑context reduction—to enable truly long‑context processing.
Mixture of Expertslong context
0 likes · 13 min read

This article examines whether the rapid growth of large‑language‑model context windows can eliminate the need for retrieval‑augmented generation, presenting experimental needle‑in‑a‑haystack tests, analysis of model performance across token lengths and needle positions, and practical guidance using an open‑source evaluation tool.
AILLMNeedle-in-a-Haystack
0 likes · 13 min read

Google’s Gemini 1.5, a new multimodal Mixture‑of‑Experts model, supports up to a million‑token context (10 million internally), can understand text, video, audio and code, learns a new language from a single prompt, and is already being used by Samsung, Jasper and Quora, positioning it as a direct challenger to OpenAI’s flagship models.
Gemini 1.5Google AILLM
0 likes · 7 min read

BladeLLM, Alibaba Cloud’s large‑model inference engine, pushes the limits of LLMs by supporting ultra‑long context lengths up to 70 K tokens, leveraging novel RaggedAttention and a DNN‑based AutoTuner to deliver superior performance, memory efficiency, and low‑latency inference across diverse workloads.
AI infrastructureAutoTunerLLM inference
0 likes · 11 min read


ITPUBMar 22, 2023 · Artificial Intelligence
OpenAI’s GPT‑4 arrives with multimodal vision, a dramatically longer context window, higher exam scores, Socratic prompting, improved safety, and new partnerships, while still in research mode and subject to bias and code‑trust limitations.
AI safetyGPT-4large language model
0 likes · 7 min read
