Tagged articles
Training Efficiency
17 articles · Page 1 of 1
A systematic study of reinforcement‑learning post‑training for large language models shows that most RL gains are concentrated in a few middle Transformer layers, and training just one such layer can match or surpass full‑parameter RL across seven models, three RL algorithms, and multiple task domains, leading to simple yet effective training strategies.
RL fine‑tuningTraining Efficiencylarge language models
0 likes · 17 min read

This article provides a practical, data‑driven comparison of Full Fine‑Tuning, LoRA, and QLoRA for adapting 7B‑70B open‑source LLMs, detailing memory requirements, training speed, cost, performance trade‑offs, step‑by‑step workflows, code examples, evaluation metrics, common pitfalls, and optimization tips to help engineers choose the most suitable fine‑tuning approach for their data and budget.
Full Fine-tuningGPU memoryLoRA
0 likes · 24 min read

The article presents Token Superposition Training (TST), which temporarily averages embeddings of non‑overlapping token bags and predicts groups of tokens in a first phase before reverting to standard token‑wise prediction, achieving up to 2.5× pre‑training speedup on 10B‑1B MoE models without altering model architecture or inference.
LLM PretrainingMCE lossMulti-Token Prediction
0 likes · 9 min read

The RAEv2 framework replaces traditional VAEs by summing multiple layers of pretrained vision encoders, combines RAE with REPA for complementary semantic and spatial gains, and leverages free guidance, achieving up to ten‑fold faster convergence, higher image quality, and lower compute on ImageNet‑256 diffusion training.
Image GenerationRAEv2Representation Autoencoder
0 likes · 11 min read

Nous Research introduces Token Superposition Training, which groups tokens into bags, averages their embeddings, and predicts token groups without altering model architecture or adding compute, achieving up to 2.5× faster pre‑training while maintaining standard inference.
LLM PretrainingMCE lossMoE
0 likes · 10 min read

DataFlex, built on LLaMA‑Factory, offers a unified, reproducible infrastructure that dynamically selects, mixes, and re‑weights training data, turning data into a controllable optimization object and delivering measurable gains in training efficiency and model performance for large‑scale AI models.
Data-Centric AIDataFlexDynamic Data Training
0 likes · 14 min read


AIWalkerMar 16, 2026 · Artificial Intelligence
Beyond-Hungarian replaces the costly Hungarian assignment in DETR with a differentiable, query‑free matching scheme that halves training latency, boosts large‑object AP by 4.2 points, and introduces a GT‑Probe module and dual‑loss framework, while detailing trade‑offs, ablations, and future challenges.
DETRGT-ProbeHungarian matching
0 likes · 14 min read

A new study suggests that large AI models could reduce their reliance on massive text corpora by early‑fusing multimodal data such as video and sensor streams, potentially slashing training costs, improving generalization, and prompting a shift toward more embodied notions of intelligence.
AI researchTraining Efficiencyembodied intelligence
0 likes · 6 min read

This article analyzes the training‑efficiency challenges of multi‑turn agentic reinforcement learning and compares four recent open‑source frameworks—AReal (Ant), Seer (Moonshot), Slime (Zhipu) and verl (Bytedance)—examining their asynchronous inference designs, rollout‑train separation, long‑context handling, off‑policy mitigation, and system‑level optimizations to guide framework selection.
Asynchronous InferenceRL SystemsTraining Efficiency
0 likes · 18 min read

This article explains DeepSeek's three core innovations—DeepSeekMoE with fine-grained experts and shared experts, Multi-Head Latent Attention (MLA) for compressed KV-cache, and Multi-Token Prediction (MTP) for faster training convergence—detailing how each reduces compute costs and memory overhead while maintaining model performance.
DeepSeekKV cache optimizationLLM Architecture
0 likes · 5 min read

dots.llm1, an open‑source 142‑billion‑parameter Mixture‑of‑Experts language model from hi lab, achieves Qwen2.5‑72B‑level performance after training on 11.2 T high‑quality tokens, and the release includes full models, intermediate checkpoints, and detailed training pipelines for the research community.
AI researchTraining Efficiencylarge language model
0 likes · 10 min read

The paper introduces Critique Fine-Tuning (CFT), a method that replaces simple imitation in supervised fine‑tuning with critique‑based learning, achieving superior reasoning performance on mathematical benchmarks using only 50 K samples, outperforming traditional reinforcement‑learning approaches that require millions of examples.
AI ReasoningCritique Fine-TuningMathematical Benchmarks
0 likes · 7 min read

The article reviews DeepSeek's open‑source R1 and V3 large language models, highlighting their technical breakthroughs, cost advantages, expert opinions, industry adoption across chips, cloud services, and applications, and discusses future directions for model scaling, distillation, and AI competition.
AI competitionAI industryDeepSeek
0 likes · 13 min read

This article analyses DeepSeek's V3 and R1 models, explaining how their innovative MoE architecture, Multi‑Head Latent Attention, low‑cost training strategies, and distributed‑training optimizations deliver high‑performance large language models while reducing GPU/NPU demand and sparking industry excitement.
AI inferenceDeepSeekTraining Efficiency
0 likes · 16 min read


AIWalkerJan 21, 2025 · Artificial Intelligence
The paper proposes a Next Patch Prediction (NPP) paradigm that groups image tokens into high‑density patches, enabling autoregressive models to predict patches instead of individual tokens, which reduces training cost to about 0.6× and improves ImageNet FID scores by up to 1.0 across models ranging from 100 M to 1.4 B parameters.
FID improvementImage GenerationLlamaGen
0 likes · 10 min read

Qwen1.5‑MoE‑A2.7B is a 2.7 billion‑parameter Mixture‑of‑Experts model that delivers performance comparable to leading 7 billion‑parameter LLMs while cutting training cost by 75% and boosting inference speed by 1.74×, and the article details its architecture, benchmarks, efficiency analysis, and deployment steps.
MoEQwenTraining Efficiency
0 likes · 13 min read

This article reviews the challenges of ultra‑large‑scale face recognition, presents existing solutions such as metric learning, PFC and VFC, and details the proposed FFC framework with dual loaders, ID groups, probe and gallery networks, plus experimental results showing its cost‑effective performance.
AIComputer VisionTraining Efficiency
0 likes · 7 min read
