LoongForge is an open‑source, Megatron‑based multimodal training framework that unifies LLM, VLM, VLA and diffusion models, runs seamlessly on NVIDIA GPUs and Baidu Kunlun XPU, and delivers 15%‑45% end‑to‑end training acceleration while scaling linearly on thousands of cards.
Baidu's Baige team introduces an extended multimodal data loader, automated ShareGPT format conversion, and offline sequence packing techniques that together double token throughput, cut SFT training time by up to six times, and improve GPU utilization and stability for large vision‑language models.
Baidu Baige’s enhanced veRL framework dramatically boosts video frame rates and resolution limits, cuts training time, reduces memory usage, and improves model accuracy by leveraging context parallelism and optimized attention on Ampere GPUs for multimodal mixed‑training scenarios.
BoltzGen, an all‑atom generative model released by MIT and collaborators, unifies protein folding and binder design with a geometric continuous representation and a flexible design language, training on multimodal datasets and demonstrating nM‑level affinity for 66% of 26 diverse targets including proteins, nanobodies, peptides and small molecules.
OmniNav introduces a unified framework for embodied navigation that simultaneously handles instruction‑goal, object‑goal, point‑goal, and frontier‑based exploration tasks using a fast visual‑language‑driven policy and a slow memory‑augmented planner, achieving state‑of‑the‑art performance and real‑world 5 Hz deployment.
The article provides a detailed analysis of Meta’s PerceptionLM, an open‑source perception language model built on Llama 3, describing its vision encoder, projector, dynamic tiling, three‑stage training pipeline, model variants, and competitive performance on image and video benchmarks.
