Beyond Bigger: Macaron‑V1 Introduces Continuous Learning and Collective Intelligence

In June 2026, Zhipu released GLM‑5.2, a large‑scale MoE model that impressed even Vercel founder Guillermo Rauch. While GLM‑5.2 is a strong foundation, the authors note that after training and parameter freezing, models struggle to keep improving during real‑world deployment. Zhipu’s roadmap therefore targets memory, continual learning, and self‑evaluation as the next technical challenges.

Mind Lab’s newly open‑sourced Macaron‑V1 pursues exactly this direction. Rather than building a new base, it upgrades to GLM‑5.2 (and a smaller Qwen‑3.6 variant), natively supports a 2 M token context, and adds the capabilities that GLM‑5.2 plans to develop: continuous learning and group intelligence . The model’s technical blog frames these as adaptation and collaboration .

From Mind Lab’s perspective, the decisive factor for the next generation of AI is not merely enlarging a single monolithic model but combining two complementary abilities: recursive self‑improvement (RSI) that generates experience after training, and multi‑agent collaboration that lets specialized models share a common infrastructure.

Benchmark results confirm the approach. On the UI4A benchmark, which tests front‑end design ability, Macaron‑V1 scores 87.7, far surpassing Claude Opus’s 4.8. The authors also report that a single Macaron‑V1‑driven Claude Code simulation produced a functional macOS‑style web page in 22 minutes, yielding a 67 KB HTML file with working calculator, photo viewer, weather, browser, and translation features.

The performance gains stem from a few small LoRA modules mounted on the frozen base. Macaron‑V1’s core architecture is a Mixture‑of‑LoRA (MoL) system that groups tasks with similar skills into the same LoRA expert, allowing them to reinforce each other, while divergent tasks are assigned to separate LoRA modules that operate independently.

Specifically, Macaron‑V1 attaches four 1 B LoRA experts to the GLM‑5.2 base:

L0 Chat : dialogue and instruction following

L1 Agent : heavy tool use and long‑range complex tasks

L2 Coding : code understanding and terminal interaction

L3 GenUI : generative UI rendering and UI‑driven operations

The flagship “Venti” variant totals 748 B parameters (744 B base + four 1 B LoRAs). A smaller “Tall” version contains 50 B parameters (35 B base + four 3.7 B LoRAs based on Qwen‑3.6).

Historically, LoRA‑based parameter‑efficient fine‑tuning (PEFT) was viewed as a cheap substitute for full‑parameter fine‑tuning. Mind Lab’s experiments overturn this view: on a trillion‑parameter MoE model (Kimi K2), LoRA‑based reinforcement learning reduces compute and communication to roughly 10 % of full‑parameter RL while maintaining stable training curves.

Further comparisons show that, under similar RL budgets, a 1.5 B model with full‑parameter RL gains 8.33 % normalized improvement, whereas a 7 B model using LoRA (r=64) with only 0.16 B trainable parameters gains 11.31 %; a 32 B model with LoRA (r=8) and 0.07 B trainable parameters gains 20.61 %. The authors conclude that the smaller the trainable fraction, the larger the relative gain when the base is strong.

Group intelligence is validated through a voting experiment. Starting from the same Qwen‑3‑30B base, a single adapter reaches 36.44 % accuracy on AIME24, while aggregating 198 diverse adapters via majority voting raises accuracy to 48.67 %. Repeating the same adapter many times only reaches 43.78 %, indicating that diverse learning trajectories provide complementary benefits beyond simple sampling.

These results support the claim that multiple LoRA specialists can coexist on a single base and be dynamically routed per task, enabling the “expand model count” strategy.

Training such systems at scale faces the classic train‑inference mismatch of reinforcement learning, where precision drift can prevent convergence. To address this, Mind Lab built two complementary systems:

MinT : a managed infrastructure that streams only adapters between actors and learners, supports millions of adapters, and handles models up to a trillion parameters.

MindForge : a training framework that integrates the harness (the runtime environment) with the model, ensuring identical conditions during rollout and production via the Harness Context Protocol (HCP).

Within MindForge, the RSI loop repeatedly generates harder seed tasks, audits generated trajectories, and solidifies successful experience back into model parameters, creating a self‑improving cycle.

For long‑context reinforcement learning, Mind Lab introduced LongStraw , which processes shared prompts without gradients, then replays response branches with back‑propagation. On a 32×H20 GPU cluster, LongStraw enabled GLM‑5.2 to perform deterministic forward and backward passes over 2,097,152 token positions, demonstrating that millions of tokens can be handled under a fixed GPU budget.

Macaron‑V1 also integrates three novel harness components:

UI4A : a generative UI layer that preserves native HTML flexibility and can import libraries from NPM or arbitrary URLs, allowing the model to produce rich interactive content without task‑specific fine‑tuning.

REPL Harness : provides a persistent Python namespace where the model can write and chain code, save verified tools, and promote them for reuse across sessions, effectively building a personal toolbox.

HCP : a unified configuration schema that bundles agent harness settings (skills, hooks, system prompts, provider configs) for seamless migration between training and serving.

In conclusion, Mind Lab has released the weights of three Macaron‑V1 variants on Hugging Face and launched an official API. While the vision of an “Experiential Intelligence Machine”—a system that continuously learns from its own experience—remains a long‑term goal, Macaron‑V1 demonstrates a concrete step: a powerful open‑source base augmented with plug‑in LoRA modules for continual learning, coordinated via a scalable infrastructure that supports millions of adapters and long‑context reinforcement learning.

References:

David Silver & Richard S. Sutton, “Welcome to the Era of Experience” (2025).

Mind Lab, “On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters”, arXiv:2606.02437.

Mind Lab, “MinT: Managed Infrastructure for Training and Serving Millions of LLMs”, arXiv:2605.13779.

Mind Lab & Fudan, “LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget”, arXiv:2607.14952.