How Inspirational Learning Turns LLM Knowledge from External to Endogenous

Recent advances in large language models (LLMs) have reshaped human‑AI collaboration, yet many capabilities still rely on external scaffolding such as agents, skills, function calling, RAG, and harnesses. The author argues that this externalization caps the performance ceiling because each new scenario requires rebuilding the workflow.

To move toward a more human‑like learning process, the paper proposes Inspirational Learning —a method that reuses past experience internally during inference. The approach consists of four steps: Analogical Retrieval , Re‑composition , Endogenous Injection , and Experience Loop . Analogical Retrieval finds structurally similar solutions from an experience bank across domains; Re‑composition reorganizes those insights; Endogenous Injection merges them into the current prompt; the Experience Loop abstracts the new reasoning chain back into the bank for future reuse.

Two concrete injection techniques implement this idea. Prompt Injection (DIN) keeps the base model frozen, identifies domain‑invariant neurons by computing token‑level means and neuron‑wise z‑scores, constructs a DIN vector from the top‑K invariant dimensions, and retrieves cross‑domain exemplars using cosine similarity and MMR. The selected exemplars are concatenated with the query and fed to the frozen model. Network‑Layer Injection (CoDA) inserts a lightweight residual adapter into intermediate hidden states. It aligns source‑domain teacher representations ("question + chain‑of‑thought") with target‑domain student representations via a mean‑square‑error loss and an MMD loss, without requiring target‑domain annotations.

Experiments evaluate Isaac on four public benchmarks that cover programming (HumanEval), multi‑hop reasoning (StrategyQA), scientific QA (ScienceQA), and research‑level coding (SciCode). Results show notable improvements: Claude reaches 100 % on HumanEval, Qwen‑3‑8B attains 82.3 % on StrategyQA, Doubao hits 98 % on ScienceQA, and several models gain consistent lifts on SciCode. Gains are larger for weaker models (e.g., Grok‑4.5 improves by 13.5 pp on StrategyQA, Gemma‑4‑31B by 12.7 pp on ScienceQA), while stronger models still see ceiling‑raising increments.

The findings demonstrate that enhancing LLMs at inference time by reusing cross‑domain experience can substantially boost task performance without scaling parameters or compute. This provides a complementary route to model improvement that parallels, rather than replaces, larger‑scale pre‑training.

References: DIN‑Retrieval (arXiv:2604.05383), CoDA (arXiv:2604.19488), systematic validation of cross‑domain context transfer (arXiv:2604.05396).