HyperQ: 64 Qubits Enable 1.1B Model to Surpass 8B Models in Zero-Shot Tasks

Quantum Computing Background

Quantum computing differs from classical computing because qubits can exist in superpositions of 0 and 1, and quantum gates perform complex state transformations. A sequence of quantum gates acting on a set of qubits forms a quantum circuit, which is measured to obtain classical readout values. This provides a computational paradigm distinct from traditional linear transformations in neural networks.

HyperQ's Dynamic Quantum Circuits

Diffusion language models generate text by iteratively denoising from heavily masked tokens, causing each token's hidden state to change across denoising steps. Prior quantum-enhanced language models used fixed, pre-designed quantum circuits that could not adapt to different tokens or denoising steps.

HyperQ introduces a lightweight hypernetwork that, for each token at each denoising step, dynamically generates a dedicated quantum circuit based on the token's current hidden state. The process involves three steps:

Generate quantum circuit: The hypernetwork outputs rotation angles, coupling strengths, and measurement directions for the quantum gates.

Execute computation: The quantum circuit transforms the information, and features are extracted from the measurement expectation values of each qubit.

Augment model: These quantum features are added to the Transformer's attention computation to aid text prediction.

This allows different tokens to use different quantum circuits, and the same token can adjust its circuit across denoising steps. Only the new quantum branch is trained; the main language model backbone remains frozen.

Scaling Quantum Circuits Efficiently

Simulating large quantum circuits is classically intractable (e.g., 64 qubits require 2^64 amplitudes). HyperQ circumvent this with two design choices:

Sparse connectivity: Each qubit connects only to 4 neighbors, preventing an explosion in connections as qubit count grows.

IQP (Instantaneous Quantum Polynomial) circuits: Only the measurement expectation values are needed, which can be computed classically with linear scaling in qubit count.

This design reduces computational cost and mitigates the barren plateau (gradient vanishing) problem common in variational quantum circuit training, as shown by gradient variance and memory/latency measurements (Figure 5). Consequently, HyperQ can train larger quantum branches on classical hardware while remaining compatible with quantum hardware.

Experimental Results

The team evaluated HyperQ on six zero-shot tasks covering commonsense reasoning, reading comprehension, and mathematical reasoning. Key findings at 64 qubits (Figure 6):

HyperQ improves the frozen backbone by 4.71 points on average.

It surpasses LoRA fine-tuning on the same backbone while using only one-tenth of the instruction-tuning data.

Performance does not monotonically increase with qubit count: at 16 qubits HyperQ underperforms the frozen backbone, but at 64 qubits it shows significant gains.

The dynamic, token-adaptive nature is crucial: compared to the best fixed quantum circuit, HyperQ gains 3.68 points; compared to discrete structure search (which selects from a limited candidate set per token), it gains 2.00 points (Figure 7 left).

At 16 qubits, HyperQ allocates entanglement based on token information content: content words and rare subwords retain the densest couplings, while 30.2% of punctuation tokens disable internal rotations and couplings entirely (Figure 7 middle).

At 64 qubits, approximately 90% of tokens use a quantum circuit structure unique to that token, whereas fixed circuits force all tokens to share one structure, and discrete search only picks from a small predefined pool (Figure 7 right).

Quantum Cluster Deployment

Beyond classical simulation, the researchers deployed the trained quantum circuits on IBM's 156-qubit quantum cluster without modifying model weights. Despite hardware noise, most performance is retained: the six-task average score drops only 1.12 points relative to classical simulation (Figure 8). This demonstrates that HyperQ's circuits are executable on real quantum hardware.

Conclusion

HyperQ opens a new direction where each token invokes a bespoke quantum circuit, making computational structure a new dimension for model scaling. The work bridges quantum computing and large language models, and the team plans to continue exploring quantum foundation models.

Paper: Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models (arXiv:2609.24657)