BestHub
Sign in
Tagged articles

model merging

7 articles · Page 1 of 1
Machine Heart
Machine Heart
Nov 1, 2026 · Artificial Intelligence

Road MOPD: Harmonizing Conflicting Expert Teachers in Multi-Domain Distillation

JD Retail and Nanjing University propose Road MOPD, a multi-teacher online policy distillation method that uses cross-teacher feedback on the same student response to strengthen consensus signals, weaken divergent signals, and project away conflicting gradient components, enabling simultaneous improvement across high-conflict domain pairs like math+medical and code+medical on Qwen3 models.

Qwen3cross-teacher feedbackdomain adaptation
0 likes · 15 min read
Road MOPD: Harmonizing Conflicting Expert Teachers in Multi-Domain Distillation
DeepHub IMBA
DeepHub IMBA
Jun 17, 2026 · Artificial Intelligence

How a 1.5B Parameter Model Can Add External Knowledge to Any Frozen LLM

The article analyzes MEMO, a framework that equips a frozen large language model with a lightweight 1.5B‑parameter memory model fine‑tuned on a target corpus, detailing its architecture, five‑step data synthesis pipeline, structured inference protocol, experimental advantages over RAG and fine‑tuning, as well as its limitations and future research directions.

Knowledge IntegrationLLMRAG
0 likes · 19 min read
How a 1.5B Parameter Model Can Add External Knowledge to Any Frozen LLM
Machine Heart
Machine Heart
Jun 7, 2026 · Artificial Intelligence

FusionRoute: Token-Level Expert Routing and Self-Correction for Multi-LLM Collaboration

FusionRoute introduces a token‑level routing framework that dynamically selects the most suitable expert LLM for each token and adds a complementary generation step, enabling fine‑grained, stable multi‑model collaboration that outperforms existing sequence‑level and expert‑selection methods across diverse benchmarks.

AI ResearchLarge Language ModelsSelf-Correction
0 likes · 11 min read
FusionRoute: Token-Level Expert Routing and Self-Correction for Multi-LLM Collaboration
Machine Heart
Machine Heart
May 7, 2026 · Artificial Intelligence

OrthoReg: Simple Orthogonal Regularization to Eliminate Model Merging Conflicts

The paper introduces OrthoReg, a lightweight orthogonal regularization added during fine‑tuning that provably enforces weight orthogonality, thereby resolving conflicts in model merging and providing a theoretical explanation for the success of task arithmetic.

Deep LearningOrthoRegOrthogonal Regularization
0 likes · 12 min read
OrthoReg: Simple Orthogonal Regularization to Eliminate Model Merging Conflicts
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 18, 2026 · Artificial Intelligence

Model Ability Gets Squeezed Out in Multi‑Task Learning—How ESM Preserves It (CVPR 2026)

The paper reveals that multi‑task models suffer performance drops because tasks compete for the same internal subspace, and introduces Essential Subspace Merging (ESM) which separates critical directions and uses Polarized Scaling to keep multiple abilities stable, achieving significantly lower degradation than traditional baselines.

ESDESMessential subspace
0 likes · 16 min read
Model Ability Gets Squeezed Out in Multi‑Task Learning—How ESM Preserves It (CVPR 2026)
AI2ML AI to Machine Learning
AI2ML AI to Machine Learning
Nov 3, 2025 · Artificial Intelligence

Smol Training Playbook: Secrets to Building World-Class LLMs

The article details the SmolLM3 3B‑parameter model, its architecture, dual‑mode inference, a three‑stage data‑curation strategy, rigorous ablation methods, preference optimisation (APO/DPO), model merging, and practical training‑stability tricks, offering a comprehensive guide for building high‑performing large language models.

APOContext ScalingLLM Training
0 likes · 13 min read
Smol Training Playbook: Secrets to Building World-Class LLMs