AgenticSciML: Collaborative Multi‑Agent System for Emergent Discovery in Scientific Machine Learning

Overview

Scientific Machine Learning (SciML) combines data‑driven inference with physical modeling, but designing effective architectures, loss functions, and training strategies still relies heavily on expert trial‑and‑error. The paper proposes AgenticSciML , a collaborative multi‑agent system that discovers new SciML modeling strategies through structured debate, retrieval‑augmented memory, and evolutionary search.

Background: Bottlenecks in SciML Model Design

SciML methods such as Physics‑Informed Neural Networks (PINNs), neural operators (FNO, DeepONet), and domain‑decomposition approaches excel across fluid dynamics, inverse problems, and material science, yet they suffer from high manual intervention in four key design dimensions: architecture selection, physical constraint formulation, loss‑function design, and training‑strategy tuning. Existing automation (AutoML/NAS, single‑agent LLMs, symbolic regression, evolutionary search) either explores only predefined spaces or lacks iterative refinement.

AgenticSciML Framework

Three‑Stage Workflow

Structured User Input : Users provide four files— Problem.md (problem description), Requirements.md (framework, library, hardware constraints), Evaluation.md (metrics), and optional Data_config.json (data paths).

Data Analysis & Evaluation Contract : A multimodal Data Analyst generates exploratory Python code and a textual report; an Evaluator produces a unified evaluate.py script and guidelines.md for consistent scoring.

Evolutionary Solution Search : The core loop iterates over (a) selector ensemble voting, (b) knowledge‑base retrieval, (c) structured N‑round Proposer‑Critic debate, (d) Engineer code synthesis, (e) Debugger error fixing, (f) evaluation, and (g) Result Analyst reporting. The process repeats until a fixed iteration budget is exhausted.

Agent Roles

Proposer (Gemini 2.5 Pro/Flash): Generates detailed reasoning before proposing a concrete solution.

Critic (GPT‑5 Mini): Challenges the Proposer’s reasoning and suggests alternatives.

Engineer (Claude Haiku 4.5): Translates the final proposal into runnable Python code.

Debugger (GPT‑5 Mini): Iteratively fixes runtime errors.

Retriever (Gemini 2.5 Pro/Flash): Searches a curated knowledge base of 70 SciML entries.

Data Analyst (Gemini 2.5 Pro/Flash): Performs exploratory data analysis.

Result Analyst (Gemini 2.5 Pro/Flash): Produces multimodal analysis reports.

Evaluator : Generates the evaluation contract.

Selector Ensemble (GPT‑5 Mini, Grok‑4 Fast, Gemini): Votes for parent solutions to balance exploitation and exploration.

Key Mechanisms

Structured N‑Round Debate forces the Proposer to “think out loud” for the first N‑2 rounds, after which the Critic refines the reasoning, leading to a synthesized implementation in the final rounds.

Ensemble‑Guided Parent Selection always keeps the best solution (exploitation) and adds 1–2 diverse candidates chosen by majority vote among the three selector agents (exploration).

Evolutionary Tree Search organizes the solution space as a tree where each node represents a candidate implementation. Nodes generate up to ten children before being retired, preventing over‑development of a single branch.

A solution strategy is defined as an emergent discovery when it does not appear in any knowledge‑base entry but is synthesized by the agents from retrieved techniques, problem structure, and prior experiments.

Experimental Evaluation

Six benchmark problems covering function approximation, Poisson on an L‑shaped domain, Burgers’ equation, inverse operator learning, multi‑input operator learning, and sparse reconstruction of cylinder flow were used. All experiments ran on an NVIDIA A6000 GPU.

Performance gains (error reduction factors) reported from the paper’s Figure 1:

Discontinuous function approximation: ~194×

L‑shaped Poisson: ~10×

Burgers: ~11,000×

Inverse operator, multi‑input operator, and flow reconstruction: >100× (estimated from bar chart)

Key emergent strategies included:

MoE with learnable sigmoid gates for discontinuous functions.

Domain decomposition with importance sampling for L‑shaped Poisson.

Three‑stage training (pre‑train BC/IC → gPINN + adaptive weighting → RAR + L‑BFGS) for Burgers.

Linear bias‑free branch networks enforcing operator linearity.

Extending 1‑D inputs to 2‑D spatio‑temporal grids and hard BC/IC enforcement for operator learning.

Knowledge‑base ablation on the discontinuous function task showed that a full knowledge base yields the best error, while removing the KB degrades performance by ~2.3× and a random KB degrades it by ~20.7×, confirming the importance of relevant retrieved knowledge.

Cost and Resource Analysis

LLM API cost per full experiment ranged from $2.00 to $11.30, with the Proposer contributing 36‑50 % of total token usage. GPU training time dominated overall runtime (e.g., Poisson: 5.6 h GPU vs 1.7 h LLM). Human input accounted for less than 0.3 % of total generated text, demonstrating high autonomy.

Implementation Details

Core Algorithm (pseudocode)

for t = 1, 2, ..., T_max do
    P ← {best solution}               # exploitation
    P ← P ∪ SelectorEnsemble(T, K‑1)   # exploration via voting
    for each parent p ∈ P do           # parallel mutation
        kb ← Retriever(p, T, KB)       # retrieve 0‑1 relevant KB entries
        ctx ← A.get(p)                 # analysis report context
        for round r = 1 to N do        # N‑round structured debate
            if r ≤ N‑2: reasoning + critique
            if r = N‑1: synthesis + evaluate
            if r = N:   finalize proposal
        end for
        c ← Engineer(p.code, proposal)   # code generation
        c ← Debugger(c) until success      # debugging
        score_c ← Execute(c, eval.py)       # evaluation
        a_c ← ResultAnalyst(c, score_c)    # analysis report
    end for
end for
return best solution on tree               # champion

Usage Workflow

Prepare the four input files ( Problem.md, Requirements.md, Evaluation.md, optional Data_config.json).

Run the system; it automatically creates data‑analysis reports, evaluation contracts, and a baseline solution ( solution_0).

The evolutionary loop generates candidate solutions, evaluates them, and records the full evolution tree.

Final output includes the champion Python script, trained checkpoints, visualizations, and a comprehensive analysis report.

Key Visuals

Figure: Three‑stage framework with agent responsibilities.

Figure: Structured debate process.

Figure: Multi‑agent system vs. single‑agent baseline across six benchmarks (log‑scale).

Figure: Consistency of three selector agents in top‑3 nominations.

Figure: Text contribution per agent (excluding Engineer and Selector).

Conclusions and Future Directions

The study demonstrates that collaborative multi‑agent reasoning can synthesize novel SciML strategies absent from existing literature, delivering orders‑of‑magnitude error reductions at modest LLM API cost. Limitations include dependence on the knowledge‑base coverage, the need for stronger physics‑grounded verification, and computational overhead of evolutionary search. Future work aims to integrate classic solvers, richer physics‑grounded signals, hierarchical meta‑agents for adaptive coordination, expansion to multi‑physics and data‑assimilation workflows, and open‑source LLM alternatives for cost‑effective deployment.

Selected References

Raissi, Perdikaris & Karniadakis, “Physics‑informed neural networks,” J. Comput. Phys. 378, 2019.

Li et al., “Fourier neural operator for parametric PDEs,” arXiv 2010.08895, 2020.

Lu, Jin, Pang, Zhang & Karniadakis, “Learning nonlinear operators via DeepONet,” Nat. Mach. Intell. 3, 2021.

Swanson et al., “The Virtual Lab of AI agents designs new SARS‑CoV‑2 nanobodies,” Nature 646, 2025.

Lu et al., “The AI Scientist: Towards fully automated open‑ended scientific discovery,” arXiv 2408.06292, 2024.