Spectral Division of Labor: How HINTS Blends Jacobi and DeepONet for Uniform PDE Convergence

1. Introduction

The paper HINTS: Hybrid Iterative Neural‑Operator and Traditional Solver (Nature Machine Intelligence, 2024) proposes a pragmatic way to embed neural operators into existing PDE solvers. Instead of repairing the spectral bias of neural networks, the authors treat it as a feature: DeepONet excels at low‑frequency error components, while classical relaxation methods (Jacobi, Gauss‑Seidel) quickly damp high‑frequency modes.

2. Spectral characteristics of the two components

Jacobi/Gauss‑Seidel – For the damped Jacobi iteration the eigenvalues of the iteration matrix show rapid decay for high‑frequency modes and almost no decay for low‑frequency modes, which explains the classic “low‑frequency stagnation” and the motivation for multigrid.

DeepONet – Empirical studies (Rahaman et al., 2019) show that neural networks preferentially fit low‑frequency components. A table in the original paper (reproduced here in prose) summarizes the performance:

Low frequency: DeepONet reduces error dramatically; Jacobi hardly changes it.

Mid frequency: Both methods provide modest improvement.

High frequency: DeepONet may even worsen the error, while Jacobi eliminates it quickly.

The key insight is that the two methods are complementary across the spectrum.

3. HINTS algorithm

The algorithm alternates a DeepONet iterator (responsible for low‑frequency correction) with a Jacobi iterator (responsible for high‑frequency correction) according to a fixed ratio n_r (e.g., 1 DeepONet step every 15 Jacobi steps). The overall workflow is extremely simple and can be inserted into any existing solver with a single if‑else branch.

HINTS‑Jacobi algorithm flow
 ============================
 Input: matrix A^h, RHS f^h, trained DeepONet G_θ
       ratio n_r, tolerance ε
 v^h ← 0
 for k = 1,2,…
    r^h ← f^h – A^h·v^h          # residual
    if ‖r^h‖ < ε then STOP
    if n_r divides k then
        # DeepONet step (low‑frequency)
        r(x) ← convert r^h to function
        δv(x) ← G_θ(k(x), r(x))
        δv^h ← evaluate δv on Ω^h
    else
        # Jacobi step (high‑frequency)
        δv^h ← ω D⁻¹·r^h
    v^h ← v^h + δv^h

Key parameters are the DeepONet‑to‑Jacobi ratio (typically 1/15 ≈ 4‑7 %) and the damping factor ω (chosen per dimension: 2/3 for 1D, 0.8 for 2D, 0.9 for 3D).

4. Numerical experiments

4.1 1‑D Poisson – Three configurations were tested: (M1) pure Jacobi, (M2) one DeepONet initialization followed by Jacobi, and (M3) HINTS with ratio 1/25. Only HINTS (M3) reached machine precision in ~200 iterations; M2 merely shifted the convergence curve without changing its slope.

4.2 1‑D Helmholtz (indefinite system) – Pure Jacobi and DeepONet‑initialization both diverged. HINTS with ratio 1/15 converged after ~300 iterations, demonstrating that DeepONet can suppress the low‑frequency amplification that causes divergence.

4.3 Higher‑dimensional and irregular geometries – Experiments on a 2‑D L‑shaped Poisson problem (unstructured triangular mesh) and a 3‑D Helmholtz problem showed that a DeepONet trained on a uniform grid can be interpolated onto the irregular mesh and still achieve uniform convergence.

4.4 Integration with multigrid (HINTS‑MG) – Replacing the standard Gauss‑Seidel smoother in a V‑cycle with HINTS reduced the required V‑cycles from ~15 to ~5 for 1‑D Poisson and turned a diverging 2‑D Helmholtz multigrid run into a convergent one within ~3 V‑cycles.

4.5 Large‑scale preconditioning – In a 3‑D cylindrical Helmholtz problem (degrees of freedom up to 10⁶) HINTS‑MG was used as a PETSc preconditioner for F‑GMRES. Compared with algebraic multigrid (AMG), HINTS‑MG kept iteration counts constant across mesh refinements and achieved lower wall‑clock times for high wave numbers, although its overhead was larger on very small problems.

5. Generalization capabilities

The authors evaluate three axes of generalization:

Across discretizations : A DeepONet trained on 30 uniform nodes can be interpolated to 32‑node or unstructured meshes without loss of convergence speed.

Across input distributions : Training on Gaussian random fields (GRF) with a long correlation length still yields good performance on short‑correlation inputs, albeit with slightly slower convergence.

Across domains and boundary conditions : Changing the computational domain (e.g., from a unit square to an L‑shaped domain) or Dirichlet boundary types does not require retraining.

These results indicate that a single offline‑trained DeepONet can serve many downstream PDE configurations.

6. Implementation details (Python)

The open‑source repository github.com/kopanicakova/HINTS_precond provides two implementations: a NumPy/PyTorch prototype ( HINTS_numpy) and a PETSc‑based large‑scale version ( HINTS_petsc). The NumPy version contains the core loop shown above, a damped Jacobi solver that pre‑computes M⁻¹N and M⁻¹f, and a DeepONet that operates on the residual rather than the original RHS. Two crucial tricks are:

Normalization of the residual (L2 norm) before feeding it to DeepONet, and denormalization after inference, ensuring convergence to machine zero.

Hard‑coded Dirichlet enforcement by multiplying the network output with the factor x(x‑1) (1‑D), x·y·(1‑x)·(1‑y) (2‑D), or the analogous 3‑D polynomial.

Configuration parameters (e.g., NUMERICAL_TO_DON_RATIO = 15, OMEGA = 2/3) are stored in configs.py, making the method easy to tune for new problems.

7. Discussion and outlook

The paper emphasizes a “defect‑driven” design philosophy: rather than fixing the spectral bias of neural operators, the bias is turned into a resource that complements classical methods. Practical advantages include plug‑and‑play integration, embarrassingly parallel Jacobi steps, and compatibility with mature software ecosystems (PETSc). Limitations noted are the focus on linear PDEs, the cost of the initial DeepONet training, and the need for manual ratio tuning. Future directions include extending HINTS to nonlinear PDEs, swapping DeepONet for other operator architectures (FNO, transformer‑based), and combining HINTS with domain‑decomposition techniques.

8. Conclusion

HINTS demonstrates that a modest, low‑frequency correction from a neural operator can dramatically improve the robustness and speed of classical iterative solvers, even turning divergent schemes into convergent ones, while requiring only a tiny fraction of additional computational effort.