Collection size

96 articles

Page 3 of 5

Jun 9, 2025 · Artificial Intelligence

AI Research Highlights: Robo-DM, DeepKD, LLM Security, and Reasoning Innovations

This roundup presents recent AI breakthroughs, including Robo‑DM’s efficient robot dataset management, DeepKD’s decoupled knowledge‑distillation trainer, a novel informed white‑box attack exposing weaknesses in LLM alignment defenses, the RePPL hallucination detector, Self‑GIVE’s associative reasoning framework, and LLM‑driven RL ensemble methods.

AIknowledge distillationreasoning

0 likes · 15 min read

AI Research Highlights: Robo-DM, DeepKD, LLM Security, and Reasoning Innovations

Machine Heart

May 30, 2026 · Artificial Intelligence

How Abstract Symbols Cut AI Inference Cost by 11×

The article examines IBM Research's Abstract‑CoT approach, which replaces verbose natural‑language chain‑of‑thought reasoning with a compact abstract token vocabulary, achieving up to an 11‑fold reduction in inference tokens while maintaining comparable accuracy across math, instruction‑following, and multi‑hop QA benchmarks.

AI inferenceAbstract-CoTChain of Thought

0 likes · 11 min read

How Abstract Symbols Cut AI Inference Cost by 11×

Xiaohongshu Tech REDtech

Jul 10, 2026 · Artificial Intelligence

PIPO: Pair‑In Pair‑Out Architecture Cuts Input Length in Half and Doubles Token Output for LLMs

The PIPO (Pair‑In, Pair‑Out) architecture compresses two input tokens into one latent vector and expands each hidden state to emit an extra draft token, halving effective input length and doubling per‑step output, achieving up to +7.15 pass@4 improvement and 2.64× TTFT / 2.07× TPOT speedups on benchmarks such as AIME 2025, GPQA‑Diamond, LiveCodeBench v6 and LongBench v2 with Qwen3.5‑4B/9B backbones.

Multi-Token PredictionSpeculative Decodingefficiency

0 likes · 14 min read

PIPO: Pair‑In Pair‑Out Architecture Cuts Input Length in Half and Doubles Token Output for LLMs

Code Mala Tang

May 31, 2026 · Artificial Intelligence

Top 10 AI Papers This Week: SkillOpt, Agent Distillation, and Sleeping LLMs

This roundup reviews ten recent AI papers covering SkillOpt’s treat‑SKILL.md as trainable parameters, compiling whole agent pipelines into model weights, decentralized AI scientist teams, adding a "sleep" consolidation phase to LLMs, interface‑only fixes for frozen agents, reuse‑aware context‑cost strategies, evaluating AI’s ability to forecast scientific breakthroughs, agent aging benchmarks, the trade‑offs of complex harnesses, and multilingual food‑embedding models.

AI agentsAgent AgingAgent Distillation

0 likes · 18 min read

Top 10 AI Papers This Week: SkillOpt, Agent Distillation, and Sleeping LLMs

Machine Learning Algorithms & Natural Language Processing

Apr 8, 2026 · Artificial Intelligence

Dissecting Gemma‑4’s Architecture and Training Choices: A Technical Comparison with Qwen‑3 and GLM‑5

This article breaks down every architectural and training decision behind Gemma‑4—KV sharing, p‑RoPE, per‑layer embeddings, and a dual‑path MoE + dense MLP—while contrasting its efficiency and performance with Qwen‑3 and GLM‑5 across benchmarks, quantization strategies, and RL pipelines.

GLM-5Gemma 4LLM architecture

0 likes · 23 min read

Dissecting Gemma‑4’s Architecture and Training Choices: A Technical Comparison with Qwen‑3 and GLM‑5

DataFunTalk

Dec 24, 2021 · Artificial Intelligence

Large-Scale Pretrained Model Compression and Distillation: AdaBERT, L2A, and Meta‑KD

This article reviews three consecutive works from Alibaba DAMO Academy on compressing and distilling large pretrained language models—AdaBERT, L2A, and Meta‑KD—detailing their motivations, neural‑architecture‑search‑based designs, loss formulations, experimental results, and insights from a Q&A session.

AIModel CompressionNeural Architecture Search

0 likes · 10 min read

Large-Scale Pretrained Model Compression and Distillation: AdaBERT, L2A, and Meta‑KD

Kuaishou Tech

May 14, 2026 · Artificial Intelligence

Open‑Source Kwai Summary Attention (KSA): A Sequence‑Compression Mechanism for Long‑Context Inference

KSA inserts learnable summary tokens to compress KV cache by a factor of eight, enabling accurate long‑context retrieval with far lower memory and compute costs, and it consistently outperforms full‑attention and other hybrid methods on large‑scale benchmarks.

Efficient InferenceKSAKV cache reduction

0 likes · 13 min read

Open‑Source Kwai Summary Attention (KSA): A Sequence‑Compression Mechanism for Long‑Context Inference

Machine Learning Algorithms & Natural Language Processing

Feb 26, 2026 · Artificial Intelligence

Why Longer Token Chains Don't Mean Better Reasoning: Google's Deep Thinking Ratio

Google’s recent study shows that the length of a model’s token chain is negatively correlated with inference accuracy, and introduces the Deep Thinking Ratio (DTR) metric to identify truly reasoning tokens, enabling the Think@n strategy to halve compute cost without sacrificing performance.

Deep Thinking RatioLLMThink@n

0 likes · 6 min read

Why Longer Token Chains Don't Mean Better Reasoning: Google's Deep Thinking Ratio

Machine Heart

May 13, 2026 · Artificial Intelligence

Why Bigger Teachers Don’t Teach Better: Tsinghua’s On‑Policy Distillation Study

Recent research by Tsinghua and collaborators dissects On‑Policy Distillation for large language models, revealing that higher‑scoring teachers often fail to improve students unless their thinking patterns align, detailing token‑level overlap dynamics, failure cases, and two practical remedies to rescue ineffective distillation.

RL post-trainingTeacher-Student AlignmentToken Overlap

0 likes · 9 min read

Why Bigger Teachers Don’t Teach Better: Tsinghua’s On‑Policy Distillation Study

Architect's Alchemy Furnace

Feb 6, 2025 · Artificial Intelligence

How Knowledge Distillation Powers Efficient Large‑Model Deployment

This article explains how knowledge distillation enables massive AI models to be compressed and deployed efficiently, covering its principles, classification dimensions, implementation steps, innovative practices at DeepSeek, real‑world applications, and future research directions.

Artificial IntelligenceDeepSeekModel Compression

0 likes · 11 min read

How Knowledge Distillation Powers Efficient Large‑Model Deployment

Architect's Guide

May 13, 2025 · Artificial Intelligence

DeepSeek Model Distillation Technology: Overview, Innovations, Architecture, Training, Performance, and Challenges

This article provides a comprehensive overview of DeepSeek's model distillation technology, detailing its definition, key innovations, architecture, training methods, performance gains, and the remaining challenges such as the implicit performance ceiling and multimodal data distillation.

DeepSeekKnowledge TransferModel distillation

0 likes · 14 min read

DeepSeek Model Distillation Technology: Overview, Innovations, Architecture, Training, Performance, and Challenges

Machine Learning Algorithms & Natural Language Processing

May 16, 2026 · Artificial Intelligence

Token Superposition Training Accelerates LLM Pre‑training 2.5× Without Changing Architecture

Token Superposition Training (TST) speeds up large‑language‑model pre‑training by up to 2.5× without altering model architecture or compute budget, using a superposition phase that averages token embeddings into bags and predicts groups of tokens, followed by a standard recovery phase, as demonstrated on 10B‑parameter MoE and smaller models.

LLM pretrainingMCE lossMoE

0 likes · 10 min read

Token Superposition Training Accelerates LLM Pre‑training 2.5× Without Changing Architecture

Data Party THU

Aug 22, 2025 · Artificial Intelligence

TwigVLM: How Tiny Branches Accelerate Large Vision‑Language Models

TwigVLM introduces a lightweight “twig” module that prunes visual tokens early and enables self‑speculative decoding, achieving up to 154% speedup on long‑text generation while preserving 96% of original LVLM accuracy, as demonstrated on LLaVA‑1.5‑7B and other benchmarks.

LVLMSpeculative Decodingmodel acceleration

0 likes · 14 min read

TwigVLM: How Tiny Branches Accelerate Large Vision‑Language Models

Machine Learning Algorithms & Natural Language Processing

Jun 18, 2026 · Artificial Intelligence

From Imitation to Optimization: Recent Advances in On-Policy Distillation

This article surveys the latest research on On-Policy Distillation for large language models, covering methods that improve training stability, self‑distillation frameworks, and detailed analyses of when and why OPD succeeds or fails, with concrete experimental results and practical insights.

Entropy-AwareModel CompressionReinforcement Learning

0 likes · 19 min read

From Imitation to Optimization: Recent Advances in On-Policy Distillation

Baobao Algorithm Notes

May 26, 2025 · Artificial Intelligence

When Should Large Language Models Think? 10 Cutting‑Edge Strategies to Boost Reasoning Efficiency

This article reviews ten recent papers that tackle the over‑thinking problem in large language models by shortening chain‑of‑thought reasoning, introducing dynamic early‑exit, adaptive thinking triggers, and reinforcement‑learning‑based training, showing how models can maintain or improve accuracy while dramatically reducing token usage and latency.

AI researchModel Pruningadaptive inference

0 likes · 38 min read

When Should Large Language Models Think? 10 Cutting‑Edge Strategies to Boost Reasoning Efficiency

Alibaba Cloud Big Data AI Platform

Jul 23, 2025 · Artificial Intelligence

Unlock Efficient LLMs: How Alibaba’s PAI EasyDistill Powers Model Post‑Training

This article explains how Alibaba Cloud's AI platform PAI leverages the EasyDistill framework for post‑training model optimization, covering knowledge distillation concepts, data synthesis techniques, basic and advanced distillation training, the DistilQwen model family, real‑world customer cases, and step‑by‑step practical demos.

AI platformEasyDistillLLM Optimization

0 likes · 12 min read

Unlock Efficient LLMs: How Alibaba’s PAI EasyDistill Powers Model Post‑Training

Baobao Algorithm Notes

Dec 24, 2023 · Artificial Intelligence

Must‑Read AI Agent and LLM Research Papers for Deep Understanding

This curated reading list compiles essential papers on AI agents, task planning, hallucination mitigation, multimodal models, image/video generation, foundational LLM research, open‑source large models, fine‑tuning techniques, and performance optimization, providing a comprehensive roadmap for anyone aiming to master modern generative AI.

AI agentsmultimodal learningperformance optimization

0 likes · 23 min read

Must‑Read AI Agent and LLM Research Papers for Deep Understanding

Machine Learning Algorithms & Natural Language Processing

Apr 14, 2026 · Artificial Intelligence

Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix

On‑Policy Distillation (OPD) is widely used for post‑training large language models, but the sampled‑token variant often becomes unstable due to token‑level reward imbalance, teacher‑student signal mismatch on student‑generated prefixes, and tokenizer mismatches; this article analyses the bias‑variance trade‑off, identifies three root failure modes, and proposes a teacher‑top‑K local‑support‑set objective with top‑p rollout and special‑token masking that yields more stable training and better performance on both math and agentic benchmarks.

OPDlarge language modelson-policy distillation

0 likes · 32 min read

Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix

Architects' Tech Alliance

Feb 16, 2025 · Artificial Intelligence

How DeepSeek’s Distillation Breaks Bottlenecks and Boosts Multimodal AI Performance

This article provides an in‑depth technical analysis of DeepSeek’s model distillation technology, covering its core principles, innovative data‑model fusion strategies, architecture design, training optimizations, performance benchmarks, and the remaining challenges of scaling distillation to multimodal tasks.

DeepSeekModel distillationai-optimization

0 likes · 16 min read

How DeepSeek’s Distillation Breaks Bottlenecks and Boosts Multimodal AI Performance

Machine Heart

Jul 5, 2026 · Artificial Intelligence

Tsinghua Special Award Winner Yuxian Gu Joins DeepSeek

Yuxian Gu, a 2021 Tsinghua PhD and 2025 Special Scholarship laureate, has joined DeepSeek, bringing expertise in pre‑training data selection, knowledge‑distillation for model compression, and efficient model architectures such as Jet‑Nemotron, which outperforms leading open‑source LLMs with up to 53.6× speedup on H100.

Artificial IntelligenceDeepSeekEfficient Model Architecture

0 likes · 6 min read

Tsinghua Special Award Winner Yuxian Gu Joins DeepSeek