HyperQ: 64 Qubits Let 1.1B Diffusion Model Outperform 8B Models

Researchers from Université de Montréal, Mila, and McGill University propose HyperQ , a framework that integrates quantum computing into diffusion language models by generating token-specific quantum circuits on the fly. The work is detailed in the arXiv preprint

Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

(arXiv:2609.24657, https://arxiv.org/abs/2609.24657).

Background: Quantum Computing Basics

Quantum bits (qubits) can exist in superpositions of 0 and 1, and quantum gates manipulate these states. A sequence of gates forms a quantum circuit, which is measured to produce classical outputs. This provides a computational paradigm distinct from traditional linear transformations in neural networks.

HyperQ's Dynamic Quantum Circuits

Diffusion language models generate text by iteratively denoising masked tokens, so a token's hidden state evolves across denoising steps. Prior quantum-enhanced language models used fixed, pre-designed quantum circuits that could not adapt to individual tokens or denoising steps.

HyperQ addresses this with a lightweight hypernetwork that, for each token at each denoising step, dynamically generates a dedicated quantum circuit. The process involves three steps:

Generate quantum circuit: The hypernetwork takes the token's current hidden state and outputs rotation angles, coupling strengths, and measurement bases for the quantum gates.

Execute computation: The quantum circuit transforms the information; measurement expectation values are extracted as quantum features.

Augment model: These quantum features are added to the Transformer's attention computation to aid text prediction.

This design allows different tokens to use different circuits, and the same token can adjust its circuit across denoising steps. Only the quantum branch is trained; the main language model backbone remains frozen.

Scaling Quantum Circuits Efficiently

Simulating large quantum circuits classically is challenging because a 64-qubit state requires 2⁶⁴ amplitudes. HyperQ circumvent this with two design choices:

Sparse connectivity: Each qubit connects to only four neighbors, preventing an explosion in connections as qubit count grows.

IQP (Instantaneous Quantum Polynomial) circuits: Only measurement expectation values are needed, which can be computed classically with cost scaling linearly in the number of qubits.

This approach reduces computational cost and mitigates the barren-plateau (vanishing gradient) problem common in variational quantum circuit training, enabling efficient classical training of larger quantum branches while retaining compatibility with quantum hardware.

Experimental Results

The team evaluated HyperQ on six zero-shot benchmarks covering commonsense reasoning, reading comprehension, and mathematical reasoning.

64-Qubit Performance

With 64 qubits, HyperQ improves the frozen backbone by 4.71 points on average and outperforms LoRA fine-tuning on the same backbone while using only one-tenth of the instruction-tuning data.

Circuit Size Matters, But Dynamism Matters More

At 16 qubits, HyperQ actually underperforms the frozen backbone. Scaling to 64 qubits yields significant gains, showing that raw qubit count alone is insufficient; the per-token dynamic adaptation is crucial.

Compared to the best fixed quantum circuit, HyperQ at 64 qubits gains 3.68 points .

Compared to discrete structure search (which selects from a limited set of predefined circuits per token), HyperQ gains 2.00 points .

Analysis of the learned circuits reveals:

At 16 qubits, HyperQ allocates entanglement based on token information content: content words and rare subwords retain dense coupling, while 30.2% of punctuation tokens disable internal rotations and couplings entirely.

At 64 qubits, approximately 90% of tokens use a circuit structure unique to that token, whereas fixed circuits force all tokens to share one structure, and discrete search only picks from a small candidate pool.

Quantum Cluster Validation

To test real-hardware feasibility, the researchers deployed the trained quantum circuits (without modifying model weights) on an IBM 156-qubit quantum cluster . Despite hardware noise, the six-task average score dropped only 1.12 points relative to classical simulation, demonstrating that HyperQ's circuits are executable on current quantum hardware.

Conclusion

HyperQ establishes a new direction where each token invokes a bespoke quantum circuit, making computational structure a new dimension for model scaling. The work demonstrates that quantum-augmented diffusion models can achieve strong performance with far fewer parameters, and that the approach is viable on near-term quantum devices. The team plans to continue exploring quantum foundation models.