AI Finds 2.2M Crystals But Fails Kiwi Math: The Human-AI Work Divide

Two Defining Events: Material Discovery vs. Reasoning Failure

In November 2023, Google DeepMind published in Nature that their GNoME tool discovered 2.2 million new crystal structures, with 380,000 predicted stable — expanding known stable inorganic crystals from 48,000 to 421,000, a feat estimated to take 800 years at traditional pace (Merchant & Cubuk et al., 2023). A year later, Apple researchers tested over 20 LLMs on GSM-Symbolic, a variant of the GSM8K grade-school math benchmark with irrelevant details added (e.g., "5 kiwis were slightly smaller"). Performance dropped up to 65%, with the same model varying 15 percentage points across 50 runs (Mirzadeh et al., 2024). The conclusion: current LLMs mimic reasoning patterns from training data rather than performing genuine logical reasoning.

How GNoME "Discovered" 2.2 Million Structures

GNoME's pipeline: (1) generate candidates via structural deformation of known crystals and random chemical-formula combinations; (2) use a graph neural network (GNN) to predict stability, replacing expensive density functional theory (DFT) calculations; (3) send only high-confidence candidates for DFT verification, feeding results back for active learning. Final discovery accuracy reached 80% versus ~50% for prior models. Crucially, the stability criteria (energy below convex hull, no spontaneous decomposition) were defined by human materials scientists; AI only executed the search within human-set boundaries.

Three Sources of AI's "Novel Insights"

AI's apparent creativity stems from three recombination capabilities: (1) massive memory linking dispersed knowledge across domains; (2) cross-domain semantic transfer in embedding space, where structurally similar practices in different fields become neighbors; (3) exhaustive enumeration of edge cases and boundary conditions that human intuition misses. Together, these recombine existing human knowledge at scale — not inventing new physics, but surfacing overlooked combinations.

The Reasoning Boundary: Mimicry, Not Logic

Apple's GSM-Symbolic study showed that even trivial irrelevant information derails models. Researchers concluded: "Current LLMs do not possess genuine logical reasoning capabilities; they attempt to reproduce reasoning steps observed in training data." This reveals a clear capability divide: AI handles short-chain, pattern-based reasoning (rule conflict checks, scenario enumeration) but fails at long-chain reasoning and implicit premise inference, often hallucinating. Most dangerously, when given a false premise, LLMs do not question it; they produce logically coherent but factually wrong outputs because they lack grounding in reality.

Three Things AI Cannot Do

Define value: Among 380,000 stable crystals, which warrant experimental resources depends on strategic goals, cost tolerance, and risk appetite — a value judgment, not a calculation.

Set boundaries: Compliance, safety, and ethical red lines are societal agreements, not statistical patterns in text.

Bear responsibility: When AI errs, no one holds the model accountable. Hallucination rates are real: best models 0.7% on summarization, worst 10%; in source attribution (CJR 2025), Grok-3 erred 94%, ChatGPT Search 67%. Domain-specific errors: legal 18.7%, scientific 16.9%, medical 15.6% (Terzo × Visual Capitalist). A 2023 New York lawyer cited six fabricated ChatGPT cases and was fined $5,000.

Therefore, all AI output must be treated as drafts requiring human verification, selection, and approval.

Four Roles Where AI Accelerates Rule-Making

Induction & extraction: Feed historical documents, cases, and tacit practices to generate a rule draft .

Gap checking: Detect clause conflicts, uncovered scenarios, and enumerate edge cases — e.g., in automotive ISO 26262 HARA analysis, AI can list boundary combinations for "sensor failure during high-speed cruise," but the functional safety engineer must judge actual hazard and ASIL level.

Rule translation: Convert natural-language business rules ("if temperature exceeds threshold for >3 seconds, trigger fallback") into structured pseudocode, state machines, or decision trees for engineering implementation — bridging the "human language to executable logic" gap.

Verification & simulation: Batch-simulate scenarios before deployment. GNoME's predictions were physically validated at Lawrence Berkeley's A-Lab: 41 of 58 target compounds synthesized successfully (>70% success), demonstrating "small-batch validation before full-scale production."

All four outputs remain drafts; humans must review each line and ask "Is the premise correct?"

The Shifting Bottleneck: From Generation to Judgment

AI makes proposal generation cheap (dozens of rule drafts in minutes), so the bottleneck moves from "can't think of options" to "can't choose the right one." Selecting the correct option requires three things AI lacks: genuine business understanding, risk judgment, and the accountability that comes with a responsible position.

Three Immediate Actions

Classify your tasks: rule-setting (human) vs. rule-execution (AI).

Use AI for drafts only; keep selection authority. Have AI list candidates, you pick, then ask AI to deepen the chosen ones — never accept a one-shot finished product.

Verify every premise you feed the model; it will not brake on false premises — you must be the brake.

Conclusion

GNoME opened the search space from thousands to millions, but could not tell researchers which 380,000 crystals were worth three months of lab work. DeepMind released the list to the community because that judgment inherently belongs to humans. AI's value: expanding your candidate list from 10 to 1,000. Your value: picking the right 1 from 1,000. AI opens possibilities; you hold the line.