Machine Heart

Oct 31, 2026 · Artificial Intelligence

NVIDIA's PixelUMM: Encoder-Free Unified Image & Video Understanding in Pixel Space

NVIDIA and University of Waterloo introduce PixelUMM, an encoder-free unified multimodal model that processes images and videos directly in pixel space using patchified representations and a Mixture-of-Transformers backbone, achieving competitive performance across understanding and generation benchmarks without VAE or ViT.