Machine Heart

Apr 24, 2026 · Artificial Intelligence

Audio-Omni: A Unified Multimodal Model for Understanding, Generating, and Editing Audio Across Sound, Music, and Speech

Audio-Omni, a unified multimodal audio model presented at SIGGRAPH 2026, combines a frozen large multimodal language model with a trainable diffusion generator to achieve state‑of‑the‑art understanding, generation, and instruction‑based editing across general sounds, music, and speech, leveraging a million‑scale AudioEdit dataset and a hybrid conditioning architecture.