HY‑WU: Real‑Time Adaptive AI Model That Generates Parameters On‑The‑Fly

Background and Motivation

Most machine‑learning systems assume that once a model is trained its parameters remain fixed during inference. This paradigm has driven progress for over a decade, relying on larger models, more data, and longer training. However, as AI moves into more complex, heterogeneous application environments, a single static parameter set becomes a limitation because real‑world tasks are highly diverse and may even conflict with each other.

Traditional solutions such as domain adaptation or fine‑tuning require additional training and increase deployment complexity. The authors therefore ask whether real‑time adaptation is possible.

HY‑WU Proposal

The paper "HY‑WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text‑Guided Image Editing" introduces a framework that generates model parameters on the fly during inference. Instead of relying on a fixed parameter set, the system produces a task‑specific set of parameters for each input, enabling a single base model to exhibit multiple behaviors.

System Architecture

HY‑WU consists of three stages:

Condition Extraction : Features are extracted from the input image and the textual editing instruction, then fused into a unified condition representation.

Parameter Generation : A Transformer‑based network takes the condition representation and outputs a new set of parameters in the form of LoRA adapters.

Image Editing : The generated LoRA adapters are injected into the base model, which then performs the editing task with the newly created parameters.

This design allows the same base model to adapt its behavior for each distinct task.

Training Strategy

Unlike conventional approaches that pre‑train a large model and later learn to reconstruct its parameters, HY‑WU trains the parameter‑generation network directly. The training loop follows five steps: (1) input image and instruction, (2) generate task‑specific parameters, (3) produce the edited image, (4) compute loss against the target image, and (5) update the parameter‑generation network. This end‑to‑end optimization avoids storing multiple full models and reduces training complexity.

Experimental Evaluation

Human Evaluation : Large‑scale crowdsourced judging compared HY‑WU against several strong baselines. HY‑WU achieved win rates of 78.4% over Step1X‑Edit, 70.5% over Qwen‑Image‑Edit, 68.3% over LongCat‑Image‑Edit, and 55.5% over FLUX.2. Against commercial systems, it scored 55.6% over Seedream 4.5 and 55.5% over GPT Image 1.5, only slightly trailing the top‑tier Nano Banana series.

Automatic Evaluation (WU‑Eval) : The authors built an automatic metric covering instruction alignment, content consistency, structural rationality, and image quality. HY‑WU obtained the highest overall score of 4.27 (consistency 4.13, structure 4.30, quality 3.98). Compared with the strongest open‑source model, consistency improved by ~0.27 and structure by ~0.23.

Public Benchmarks : On GEdit‑Bench, HY‑WU ranked first among open‑source models; on ImgEdit‑Bench it achieved a total score of 4.05, ranking second.

Conflict‑Task Experiments : The authors crafted contradictory editing tasks (e.g., de‑blur vs. blur, restoration vs. aging). Three strategies were compared:

Single LoRA : Separate fine‑tuned models per task performed well on their own task but could not handle others, showing over‑specialization.

Shared LoRA : A single shared adapter handled all tasks but produced compromised results (e.g., “half‑blur” when both blur and de‑blur were required).

HY‑WU : Dynamically generated parameters per input allowed each task to be executed correctly without interference, demonstrating that on‑the‑fly parameter generation resolves task conflicts.

Implications

The study shows that real‑time, inference‑stage adaptation can substantially improve performance on diverse and conflicting image‑editing tasks. By learning to generate task‑specific parameters, a single model can flexibly switch behaviors, reducing the need for repeated fine‑tuning and simplifying deployment.

From a broader perspective, HY‑WU points toward future AI systems that continuously adjust their internal parameter structures during operation, enabling them to cope with ever‑changing task distributions without costly retraining.