Skip to content
Tagged articles

OpenMOSS

2 articles · Page 1 of 1
Machine Heart
Machine Heart
Oct 31, 2026 · Artificial Intelligence

Hybrid LLM Attention: Linear or Sliding Window? Fudan OpenMOSS Finds Both Excel

Fudan University's OpenMOSS team systematically compares sliding window and linear attention mixed with full attention in long-context LLMs, revealing complementary trade-offs: sliding window excels at length extrapolation but suffers in long-context training, while linear attention shows the opposite pattern, and proposes improvements like larger windows, LongCE loss, and sliding-window linear attention with log correction for better extrapolation.

Fudan UniversityLinear attentionOpenMOSS
0 likes · 13 min read
Hybrid LLM Attention: Linear or Sliding Window? Fudan OpenMOSS Finds Both Excel
Machine Heart
Machine Heart
Aug 12, 2026 · Artificial Intelligence

A Future‑Predicting Critic Propels VLA Reinforcement Learning

The World Critic Model (WCM) augments the critic in vision‑language‑action reinforcement learning with future state prediction, enabling robots to evaluate not only the current value but also anticipate upcoming dynamics, which dramatically improves both in‑distribution and out‑of‑distribution performance across multiple benchmarks.

OpenMOSSPOMDPReinforcement Learning
0 likes · 12 min read
A Future‑Predicting Critic Propels VLA Reinforcement Learning