How JoyAI-Image-Edit Brings Spatial Intelligence to Open‑Source Image Editing

Background

Unified multimodal models aim to combine image understanding and generation, but existing open‑source models lack deep spatial reasoning, causing distortions when editing object positions, viewpoints, or scale.

JoyAI‑Image‑Edit

JoyAI‑Image‑Edit is an open‑source multimodal foundation model that jointly performs text‑to‑image generation, image understanding, and instruction‑guided editing with explicit spatial perception. It is built on the MLLM‑MMDiT unified architecture, which tightly couples visual perception and generative modules.

Core Technical Contributions

Deep fusion of perception and generation – The MLLM‑MMDiT backbone allows semantic information from the understanding branch to flow directly into the diffusion‑based generation branch, enabling geometry‑aware control.

Spatial editing paradigm – Supports three operations that were unavailable in prior open‑source models:

Viewpoint transformation: natural‑language specification of camera yaw, pitch, and zoom; the model synthesizes a new view while preserving scene geometry.

Continuous spatial roaming: generation of a coherent sequence of views along a user‑defined trajectory, effectively “walking” through the inferred 3‑D space.

Object‑level manipulation: translation, scaling, or rotation of individual objects with consistent occlusion and lighting.

Broad editing functionality – In addition to spatial operations, the model provides 15 generic editing primitives (replace, delete, add, style transfer, etc.) and demonstrates strong performance on long‑text rendering and multi‑view consistency benchmarks.

Data Engine and Training Corpus

The spatial capabilities are driven by two components:

OpenSpatial data engine – An internal pipeline that automatically synthesizes spatial annotations (camera parameters, depth, occlusion masks) for training.

Multi‑view dataset – Approximately one million view groups rendered with Blender 4.5, covering diverse scenes and object configurations. This dataset supplies the model with dense multi‑view supervision.

Performance

On publicly released benchmarks, JoyAI‑Image‑Edit achieves spatial understanding and editing scores comparable to leading closed‑source systems and surpasses existing open‑source baselines of similar scale.

Access and Usage

The model weights, inference code, and the multi‑view dataset are released on Hugging Face and GitHub. Developers can clone the repository, install the required dependencies, and run the provided inference scripts to perform text‑guided generation, image understanding, or any of the spatial editing operations.

# Example command to run spatial editing
git clone https://github.com/joyai/joyai-image-edit.git
cd joyai-image-edit
pip install -r requirements.txt
python edit.py --input image.jpg --instruction "move the chair 30 cm to the left and view from a 45° angle"

Potential Applications

Because the model can generate geometry‑consistent multi‑view images from a single input, it is applicable to e‑commerce (automatic multi‑angle product images), embodied AI (synthetic training data for robot navigation), 3‑D reconstruction (generating consistent view sequences from few images), and creative industries such as architecture, game design, and film.

Resources

Model repository: https://github.com/joyai/joyai-image-edit Model hub:

https://huggingface.co/joyai/joyai-image-edit