SenseNova-U1.5: 8B MoT Unified Multimodal Model Tops Open-Source Benchmarks

01 Traditional Multimodal Paradigm Replaced

Traditional multimodal systems use separate pathways: a pretrained visual encoder for understanding (semantic features) and a VAE for generation (latent-space pixel fidelity). These two representation spaces are disconnected. SenseNova-U1 previously proved a native unified model (NEO-unify) without external encoders or VAEs, but its per-token RGB patch reconstruction caused seam artifacts, texture breaks, and geometric inconsistencies at high resolutions.

02 From Independent Patches to Spatial Joint Reconstruction

SenseNova-U1.5 replaces per-token pixel prediction with a 2D feature field. Visual tokens are projected onto a 2D grid, then upsampled via 3×3 convolutions and Pixel Shuffle with factors 2, 2, and 8. Adjacent regions exchange information before final pixel values are fixed, so color, texture, and geometry are solved jointly. The model retains a high compression ratio (one visual token per 32×32 pixel region) but uses a lightweight spatial decoder instead of a patch-wise MLP, preserving sequence efficiency while significantly improving spatial coherence and enabling native 4K resolution support.

Table 1: SenseNova-U1.5 Configuration

03 MoT: Shared Attention with Specialized Processing

The model retains the native Mixture-of-Transformers design. Clean multimodal context and noisy visual states are interleaved in one sequence. Text tokens use causal attention; clean image blocks use bidirectional attention within the block; generation tokens attend bidirectionally within their block and to all preceding clean context. Reverse paths are masked to prevent clean representations from being polluted by random generation states. Unification does not mean full parameter sharing: understanding and generation streams keep separate attention projections, normalization layers, and feed-forward modules, dynamically routed by token type. Shared attention serves as a cross-stream communication interface while specialized parameters preserve distinct computational needs for perception and synthesis.

Training combines three objectives:

Autoregressive language modeling for semantic understanding and multimodal reasoning.

Pixel-space flow matching for direct noise-to-image learning in RGB space.

LPIPS perceptual supervision to constrain structural consistency and local visual coherence.

04 Training: Specialize First, Then Unify

SenseNova-U1.5 adopts a staged post-training strategy rather than joint optimization with a single reward.

Stage 1: Generation Pretraining (3 phases)

Phase 1: 180K steps, resolution 256²–1024².

Phase 2: 100K steps, resolution up to 1024².

Phase 3: 185K steps, resolution 512²–4096², adding image editing and interleaved generation.

Stage 2: Unified Mid-training (80K steps)

Data mixture: ~30% understanding, ~40% text-to-image, ~20% image editing, ~10% interleaved data.

Stage 3: Unified SFT (10.5K steps)

Strengthens instruction following.

Stage 4: Four-Expert Reinforcement Learning

Separate experts for:

Visual aesthetics (alternating HPSv3++ and OCR rewards).

Bilingual text rendering (PaddleOCR multi-set IoU).

Infographic generation (OCR then DPO, alternating text and aesthetics rewards).

Image editing (five-dimensional reward, taking the minimum to expose the weakest dimension and prevent high scores in one dimension from masking critical errors).

Table 2: Training Recipe

Figure 4: Post-training Pipeline

Stage 5: Multi-Expert On-Policy Distillation

The student model generates trajectories, then routes them by capability to the corresponding frozen expert. Velocity-field distillation merges the four experts into a single unified model. 800 optimization steps, 25,600 samples per domain. The understanding branch, last three generation layers, and generation output head are frozen.

This "specialize first, then unify" approach is far more reliable than a single joint reward optimization.

05 Data: Large-Scale and High-Resolution

Image generation side adds ~59 million image-text pairs from 78 sources. Crucially, ~88.2% of effective training samples exceed 1024² resolution, and ~64.4% exceed 2048². Image editing data comprises ~38 million pairs covering general editing, infographic editing, reference-conditioned editing, and spatial control editing. Interleaved data composition: ~45% lifestyle trajectories, ~29% infographics, ~19% video-derived sequences, ~8% reasoning-intensive samples. RL data includes ~280K aesthetic prompts, ~60K OCR prompts, ~120K editing samples, and ~120K DPO preference pairs.

Figure 5: Training Corpus Composition

06 Evaluation: 8B MoT Flips the Table

General Image Generation

On Qwen-Image-Bench with prompt enhancement, SenseNova-U1.5 scores 60.22 (English) and 60.13 (Chinese), the best overall among open-source models.

Table 4: Qwen-Image-Bench EN

Table 5: Qwen-Image-Bench ZH

On GenEval, it achieves an overall score of 0.92, competitive with leading closed-source models (GPT-Image-2), excelling in counting, spatial positioning, and attribute binding.

Table 6: GenEval

Table 7: GenEval2

Dense Text Rendering

On CVTG-2K, average score reaches 0.948, with 4-region and 5-region word accuracy staying above 0.95, securing the top spot.

Table 11: CVTG-2K

Editing

ImgEdit overall score 4.59, surpassing all evaluated models including Nano-Banana-Pro and GPT-Image-1. WeEdit average 7.46, highly competitive among open-source models.

Table 16: ImgEdit

Table 18: WeEdit

Interleaved Generation

On OpenING with CoT, U1.5 achieves the highest score of 9.18, with strong image-text coherence, human alignment, and multi-step consistency.

Table 21: OpenING

Understanding Not Sacrificed

On MMMU, MMBench, OCRBench the model remains competitive; language benchmarks: MMLU-Pro 86.67, IFEval 93.35. This shows generation capability does not come at the cost of understanding; the two reinforce each other.

Table 3: Understanding/Language

07 Conclusion

SenseNova-U1.5 signals a paradigm shift: multimodal unification does not require the legacy visual-encoder + VAE pipeline. Native pixel-space modeling can simultaneously support understanding, reasoning, generation, editing, and interleaved generation. Moreover, reasoning can improve generation, and generation can become part of the model's internal reasoning. The boundary between perception and creation is being erased by this natively unified architecture, and a new round of competition in visual AI has just begun.

https://arxiv.org/abs/2609.11929
GitHub: https://github.com/OpenSenseNova/SenseNova-U1
HuggingFace: https://huggingface.co/collections/sensenova/sensenova-u15