From QA to Task‑Oriented Agents: Recent Trends in Large Language Models

1. Building LLM Agent Systems from Multiple Angles (Long‑Horizon and Evolution Remain Crucial)

Recent work shifts from basic capabilities toward reliable agent actions in complex real‑world environments, emphasizing long‑sequence planning, memory management, tool integration, risk grading, and multi‑agent governance.

Multi‑Agent Collaboration and Task Allocation : examples include Agora (auction‑based task allocation), Fictional Worldbuilding (hierarchical context compression), Communication‑Efficient Digital‑Twin Coordination (heterogeneous embodied coordination), and L‑MAD (legal reasoning via multi‑agent debate).

Long‑Horizon Planning : benchmarks such as LongMedBench (medical long‑term decision), Long‑Horizon‑Terminal‑Bench, GATS (graph‑enhanced tree‑search planning), and Remember When It Matters (active‑memory agents to mitigate behavior decay).

Self‑Evolution and Reliable Agents : systems like SAGEAgent (self‑evolving modality acquisition), Shared Selective Persistent Memory, ProofCouncil/OpenProver (mathematical proof agents), Toward Auditable AI Scientists (hypothesis‑evolution protocol), and ARCANA (reflective multi‑agent program synthesis).

Trust, Safety, and Explainability : approaches such as ConceptSMILE (concept‑level XAI trust auditing), TrustX Agent Risk Classification, Scoped Verification (robust context evolution under distribution shift), and Multimodal Reward Hacking.

2. Test‑Time Scaling / Inference‑Time Optimization

Inference Enhancement : KV‑PRM (process reward modeling via KV‑cache transfer), CogniConsole (formal abstraction for inference‑time control), and various CoT/ToT/Monte Carlo reasoning improvements.

Efficiency : quantization, sparsity, MoE service optimization, context extension techniques like Jet‑Long, and broader test‑time scaling strategies.

Benchmarks and Evaluation : MedRealMM (real‑world multimodal medical benchmark) and new long‑context/agent benchmarks that prioritize robustness over average performance.

Key papers cited include:

Self‑Guided Test‑Time Training for Long‑Context LLMs – test‑time self‑training without extra data.

Test‑Time Scaling for Small VLMs on Multilingual MCQ – small models gain performance via inference‑time scaling.

KV‑PRM: Process Reward Modeling via KV‑Cache Transfer – enables scaling of multi‑agent inference.

CogniConsole: Inference‑Time Control as Formal Abstraction.

LLM Routing with Contextual Bandits – reinforcement‑learning‑based dynamic routing of different LLMs.

3. Foundation Models and Multimodal Directions

Hot topics include:

Self‑Compacting Language Model Agents – models decide when and how to compress context.

Training‑time Augmentation – data augmentation, synthetic trajectories, and RL to boost agent abilities.

Manifold Bandits – treating sampling as structured bandits in the latent manifold of LLM representations.

GLM‑5 – a new foundation model.

Multimodal advances:

LiveEdit – diffusion‑based framework for real‑time streaming video editing, addressing causality, latency, and fidelity.

Vera – training‑free framework for long‑context vision‑language models, introducing VER heads that retrieve visual evidence during high‑entropy moments.

Unlimited OCR – Baidu’s 3B‑parameter MoE vision‑language model for one‑shot long‑horizon parsing, using Reference Sliding Window Attention for constant KV cache.

UniverSat – resolution‑ and modality‑agnostic transformer for Earth observation, mapping arbitrary spatial, spectral, and temporal patches into a shared embedding space.

4. MOE Integration Trends

Director : online proactive expert placement, enabling distributed MoE service acceleration (INFOCOM 2026).

Sticky Routing : memory‑efficient inference via training‑time routing optimization.

iLENS : LLM‑guided explainable MoE, where the LLM itself explains MoE decisions.

5. Open‑Source Large Model Momentum

Z.ai GLM‑5.2 – ~750B MoE with 40B active parameters, 1M‑token context window, strong on SWE‑bench Pro (62.1) and Terminal‑Bench, leveraging IndexShare to cut long‑context costs.

DeepSeek V4‑Pro – 1.6T MoE supporting 1M context, achieving frontier performance on coding benchmarks.

Tencent Hy3 (295B MoE) and Liquid AI small efficient models (Raspberry Pi‑compatible) – advancing open‑source democratization.

Ant Group LingBot‑VA 2.0 – from‑scratch autoregressive pre‑training with a Semantic Visual‑Action Tokenizer, causal pre‑training, and foresight reasoning, built on a sparse‑expert MoE backbone.

Gemma 4 Technical Report – dense+MoE dual‑architecture (2.3B–31B), encoder‑free design ingesting raw audio waveforms and image patches, and a “thinking mode” that generates reasoning chains before inference, yielding strong gains on STEM, multimodal, and long‑context benchmarks.

6. Closed‑Source Large Model Advances

OpenAI GPT‑5.6 series (Sol/Terra/Luna) – released July; Sol excels in math, coding, research, reportedly proved the 50‑year‑old Cycle Double Cover Conjecture via multi‑agent systems; GPT‑Live offers full‑duplex voice interaction.

Anthropic Claude Sonnet 5 / Fable 5 – enhanced agent coding and long‑horizon tasks, competitive on multiple benchmarks, emphasizing safety and controllable agents.

Meta Muse Spark 1.1 – surpasses competitors on coding and agent tasks, low pricing (as low as $0.26 per task), and provides Meta Model API for enterprise multimodal inference.

xAI Grok 4.5 – Opus‑level performance focused on coding and agent workloads, with clear speed and cost advantages.

7. World Models as a Second Growth Curve

World models target planning, simulation, and prediction, extending agents from virtual to physical realms. Key sub‑areas include latent planning, failure detection, simulation, and various memory types (working, long, knowledge, semantic, episodic) that feed into autonomous AI systems.

8. From RAG to Knowledge OS

The emerging stack combines Ontology + Knowledge Graph + Memory + Reasoning + Planning, with ontology‑world‑model‑causal memory integration supplanting traditional Retrieval‑Augmented Generation.

9. Vertical Domain Breakthroughs

Coding and research have already surpassed human performance, driving rapid progress. In medicine (clinical SOAP note generation, pathology image retrieval, diabetic retinopathy analysis), finance, law (Ukrainian judgment drift), meteorology, hydrology, and agriculture, the volume of papers indicates LLMs and multimodal models are deeply penetrating professional sectors.

Representative domains include biology, chemistry, material science, physics, and drug discovery, with benchmarks such as LongMedBench, MedRealMM, multi‑agent trading frameworks, ProofCouncil, OpenProver, and legal/code agents.