Deflation-PINNs: Finding All Solutions of Nonlinear PDEs in a Single Training Run

Introduction

Nonlinear partial differential equations often admit multiple distinct solutions. The Landau–de Gennes model for liquid crystals exemplifies this: under identical boundary conditions, the system exhibits several stable equilibria, each corresponding to a different liquid crystal texture. Engineers need to know all existing configurations, their relative energies, and the transitions between them.

Physics-informed neural networks (PINNs) have a structural limitation for such problems: they are designed to find one solution per training run. Restarting with different random seeds may converge to the same solution or a different one, but there is no guarantee of exhaustive discovery, nor a way to verify whether all solutions have been found. This paper addresses this by embedding a "mutual separation" requirement directly into the loss function, enabling a single training to produce several separated solution branches.

Architecture: K Branch Vectors with Pairwise Repulsion

Existing operator-learning frameworks like DeepONet can represent multiple solutions, but they learn mappings between function spaces and are overkill when only a finite number of solutions is sought. The authors adapt the DeepONet branch–trunk architecture: instead of a branch network that generates coefficients from sensor points, they treat K trainable branch vectors as direct model parameters. A shared trunk network encodes spatial dependence, outputting features that are inner-producted with each branch vector to approximate the K solutions.

This reduces parameter count: for K solutions and m sensor points, DeepONet requires at least K × m parameters, while the fixed branch vectors need only K × d (where d is the branch vector dimension). Crucially, because branch vectors are now parameters, the loss function can explicitly push them apart.

A pairwise repulsion term is added to the physics-informed residual loss. For a prescribed minimum distance δ, the penalty activates when any two branches are closer than δ, averaged over all K(K-1)/2 pairs. The term is zero iff all pairwise distances are ≥ δ. The total loss is the sum of residuals across branches plus this repulsion term, weighted by a hyperparameter. Since the true minimum inter-solution distance is usually unknown, the paper provides a strategy to estimate a feasible δ from training behavior and proves the parameterization has sufficient expressive power to approximate any finite set of target functions arbitrarily well.

Validation via Energy Gradient Flow

The main benchmark uses the Landau–de Gennes theory on a square domain with non-homogeneous Dirichlet conditions, which has six stable equilibria: two diagonal states and four rotated states. This is a recognized difficult problem with a complex energy landscape where multiple minima are adjacent.

To certify that each branch corresponds to a distinct stable state, the authors do not simply compare distances to reference solutions. Instead, they construct a rigorous validation pipeline. First, they establish grid-converged reference solutions: the governing equations are discretized with second-order finite differences on 256×256 and 512×512 grids, solved with damped Newton, each state converging in three steps to residual 1e-12. Because the discrete energy functional is only first-order accurate, they apply first-order Richardson extrapolation; the resulting energies differ from literature finite-element results by only 0.06%, confirming reference reliability.

Then, each branch is used as an initial condition for the Landau–de Gennes energy gradient flow. Since gradient flow only decreases energy, its limit must be a stable state. The six branches flow to six distinct stable states. This is a dynamical criterion: a branch is associated with a state because it lies in that state's basin of attraction, not because it is spatially closer. For adjacent minima, this distinction is critical.

Three-Stage Refinement Pipeline

The paper explicitly positions the first stage as a discovery device: Deflation-PINN must locate six distinct branches in one unsupervised run. The accuracy of these branches is addressed in subsequent refinement stages.

Discovery: Single unsupervised Deflation-PINN training. Relative L2 errors range from 0.30 to 0.59.

Neural Refinement: Deflation-Deep-Ritz minimizes the energy functional. Relative

L2</sub> error drops to 0.013.</li>
<li><strong>Classical Refinement:</strong> Gradient flow and damped Newton initialized from the refined branches. Errors in three norms reach Newton solver rounding level, meaning the refined branches coincide with the reference solutions.</li>
</ul>
<img src="https://mmbiz.qpic.cn/mmbiz_jpg/RkqvU1giaESI9dicCKpPW8uOeUeYezn6C1SHSiaR42SrmNTiciaDQvvFn6vqXfk793kJkIkb2STXz3MBZjFs73VicdZ5y7jlXyURyKMEuicDxKRgT4/640?wx_fmt=jpeg&from=appmsg" alt="Figure 3: Three-stage pipeline and error magnitudes at each stage (reproduced from paper Table 1 and Sections 3.3–3.4)"/>
<p>The error distributions differ qualitatively between stages. Discovery-stage error is global: large regions of magnitude deficit appear because pointwise residual training tolerates scalar order parameters below their physical values, leaving unsaturated order parameters in bulk phases, while the director topology is already correct. After refinement, error becomes local, concentrated near the four corner defect cores—four corner squares of side 0.1 contain 59%–77% of total squared error, while regions away from defects are essentially exact.</p>
<p>This shows that the 0.30–0.59 discovery error does not mean "wrong solution"; it means "magnitude not yet accurate, topology already correct." For a stage whose purpose is identifying solution branches, this is precisely the desired state.</p>

<h2>Allen–Cahn Benchmark on Unit Disk</h2>
<p>To demonstrate generality, the authors test on an Allen–Cahn type problem on the unit disk with coefficient <code>λ=10

. This benchmark's solution set can be rigorously determined: the first two Dirichlet eigenvalues of the Laplacian are ≈5.783 and ≈14.68, and λ=10 lies between them. Analysis shows: for λ<λ₁ only the trivial solution exists; for λ₁<λ<λ₂ a unique positive radial solution exists (by moving plane method and Brezis–Oswald criterion); sign-changing solutions require λ>λ₂. Hence, for λ=10 the solution set consists exactly of the trivial solution and a pair of sign-opposite solutions— the branch count is strictly 3 . Results contrast with the Landau–de Gennes case: the three branches achieve relative L2&lt;/sub&gt; errors of <code>2.5e-5 , 1.8e-5 , and 1.2e-5 ; the trivial branch's absolute L2&lt;/sub&gt; norm drops to <code>1e-12 . On this smooth benchmark without singular structures, direct residual training already reaches reference accuracy, and no classical refinement stage is needed. The two benchmarks together delineate the method's applicability boundary: the limiting factor is not the repulsion mechanism itself, but singular structures in the solution . The corner defect cores in the Landau–de Gennes problem are structures that pointwise residuals struggle to resolve, forcing the three-stage pipeline.

Discussion: Division of Labor Between Neural Networks and Classical Solvers

Compared to recent related work, this paper redefines the role of neural networks. Most physics-informed ML aims to improve solution accuracy; here the network is assigned a task classical methods are poor at—simultaneously approximating several separated solutions without knowing how many exist or what they look like. Accuracy is handed off to Newton's method; once Newton has an initial guess inside the correct basin of attraction, convergence is rapid and reliable. This division lets each method operate in its regime of strength. Classical continuation and deflation methods also find multiple solutions but typically require good initial guesses or continuous parameter paths; randomly restarted PINNs can retry but offer no coverage guarantee. The paper's repulsion term internalizes the requirement "do not fall back to an already found solution" into the optimization objective, turning an external restart strategy into an internal parallel search. The authors list open issues: the threshold δ still relies on experience or training-behavior estimation; the branch count K must be specified a priori, while in real problems the number of solutions is often unknown; and in the presence of singular structures, discovery-stage accuracy is limited by the pointwise residual form. The second issue—unknown K —is likely the first obstacle in practical deployment. The Allen–Cahn benchmark is suitable for validation precisely because its solution count is rigorously known, a condition rarely met in applications.

Further Reading

Farrell P E, Birkisson Á, Funke S W. Deflation techniques for finding distinct solutions of nonlinear partial differential equations. SIAM Journal on Scientific Computing , 2015, 37(4): A2026–A2045.

Foundational work on deflation for PDEs; the term "Deflation" in the paper's title originates here. Classic deflation is serial—find one solution, deflate it, repeat. The paper's repulsion term is parallel—all branches push apart simultaneously in one training. This structural difference determines their respective applicable scales.

Lu L, Jin P, Pang G, Zhang Z, Karniadakis G E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence , 2021, 3: 218–229.

Original DeepONET paper, the direct architectural ancestor. The paper's Section 2.3 argues that DeepONet is designed for operator approximation (infinitely many solutions); for finite K the branch network can be degenerated to K trainable feature vectors. Assessing this degeneration requires understanding the original branch–trunk motivation and universal approximation conditions.

E W, Yu B. The Deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics , 2018, 6(1): 1–12.

Basis for the paper's second-stage Deflation-Deep-Ritz. Unlike pointwise residual minimization, Deep Ritz directly minimizes the energy functional, making it naturally suited for variational problems like Landau–de Gennes. This is key to the phenomenon in Section 3: pointwise residuals tolerate unsaturated order parameters in bulk, while the energy functional is sensitive to them, so switching to variational loss reduces error by an order and a half. The differing sensitivities of loss forms to the same physical quantity is a recurring theme in physics-informed ML.

Raissi M, Perdikaris P, Karniadakis G E. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics , 2019, 378: 686–707.

Original PINN paper; the discovery stage uses its standard residual loss. The paper's core limitation—pointwise residuals cannot resolve corner defect cores and tolerate magnitude deficits—is an inherent property of this loss form. Understanding its origin helps predict how similar issues manifest in other applications and when to switch to variational or weak-form losses.

Tancik M, Srinivasan P P, Mildenhall B, et al. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems 33 (NeurIPS), 2020.

Used in the paper's Deflation-Deep-Ritz stage (Section 3.4). The paper notes the trunk network's plain MLP is a convenience choice; Fourier features can be introduced. This relates directly to the error distribution in Section 3: refinement-stage error concentrates at the four corner defect cores, which are high-frequency structures. The spectral bias of fully connected networks favors low frequencies; this paper provides the theoretical explanation and a standard mitigation.

Reference

Disarò S, Maity R R, Bacho A. Deflation-PINNs and Deflation-Deep-Ritz: Multi-Solution Discovery, Certification, and Refinement for Nonlinear PDEs. Computer Methods in Applied Mechanics and Engineering , 2027, 464: 119476.