How GEMS Lets a 6B Open‑Source Model Beat Top Closed‑Source Image Generators

Introducing GEMS

GEMS (Agent‑Native Multimodal GEneration with Memory and Skills) is a multimodal generation framework co‑developed by teams from Shanghai AI Lab, Nanjing University, Shanghai Jiao‑Tong University, and CUHK. It equips a 6‑billion‑parameter open‑source model, Z‑Image‑Turbo, with memory and skill capabilities that dramatically improve performance on complex prompts.

Breaking the Single‑Pass Generation Paradigm

Traditional text‑to‑image models generate an image in one forward pass, which works for simple scenes but struggles with multi‑object, spatially‑constrained, or precise‑text prompts. GEMS replaces this with an iterative, multi‑agent loop that refines outputs through planning, decomposition, generation, verification, and refinement.

Multi‑Agent Loop

Planner analyzes the initial prompt and retrieves relevant expertise from a skill library to craft a stronger guiding prompt.

Decomposer breaks the complex instruction into atomic visual requirements (e.g., “red car present”, “cyber‑punk background”, “exact text spelling”).

Generator renders an initial image based on the refined prompt.

Verifier (driven by a powerful multimodal LLM such as Kimi K2.5) compares the image against each atomic requirement and outputs a binary feedback vector.

If any requirement fails, the Refiner analyzes the defect, updates the prompt, and the loop repeats, eliminating missed details.

Hierarchical Memory Engine

GEMS introduces an Agent Memory module that avoids both naïve single‑step memory and unbounded context accumulation. It stores two levels of information:

Fact‑Base Anchors : prompt, generated image, and quantified verification feedback for each iteration, kept in a compact, objective form.

High‑Level Experience : distilled from the raw reasoning traces of the multimodal LLM using a Compressor that extracts concise high‑level summaries.

The combined state tuple provides a robust long‑context foundation for the optimizer while discarding noisy redundancy.

On‑Demand Skill Library

To handle vertical, domain‑specific tasks (e.g., scientific charts, stylized illustration, precise typography), GEMS adds an Agent Skill module. A lightweight skill list is kept in memory; when a prompt matches a skill, the full skill definition is loaded and merged with the prompt before iteration.

This design mirrors software dependency management, dramatically lowering the barrier for developers or users to contribute new skills via simple Markdown specifications.

Comprehensive Evaluation

The authors benchmarked GEMS‑enabled Z‑Image‑Turbo (6B) and the 20B open‑source Qwen‑Image‑2512 across nine dimensions, including five general‑purpose benchmarks and four domain‑specific tasks.

On the GenEval2 complex‑instruction benchmark, Z‑Image‑Turbo achieved 63.5 points, surpassing the closed‑source Nano Banana 2 (44.6) and improving the normalized average score by 14.22 points over traditional single‑pass methods.

Ablation studies showed that the basic agent loop raised the score from 31.0 to 52.4; adding raw historical prompts added 3.4 points; incorporating visual context added another 3.1 points; and the hierarchical memory compressor contributed an additional 2.5 points.

In domain‑specific tests, the skill library delivered an average gain of 14.03 points, and Qwen‑Image‑2512 also saw double‑digit improvements, confirming the framework’s adaptability to different backbone models.

Moreover, GEMS reduced computational waste: on average only three images per task were needed to achieve top scores, compared to many more in heuristic‑search baselines.

Conclusion

GEMS successfully transfers the mature agent‑collaboration paradigm from language models to multimodal image synthesis, introducing a closed‑loop feedback system, hierarchical memory compression, and plug‑and‑play skill modules that together overcome the limitations of single‑pass generation, improve quality, and lower compute costs.