Why LLM Agents Rush to Call Tools and How to Stop Them

Root cause: the model is trained to call tools

When the fine‑tuning dataset is heavily skewed toward User → Tool Call → Answer (about 90% of examples), the model develops a strong reflex: any user utterance triggers a tool call.

“If the user says anything, the next step is likely a tool call.”

From a probabilistic view this is correct, so the model starts to:

Immediately call a tool when it sees keywords like “weather”.

Call a tool as soon as it detects a city name.

Even act on incomplete information.

This behavior is reinforced by the loss function that rewards tool calls.

Why prompts alone cannot stop the rush

Even if the system prompt explicitly says “only call a tool when necessary”, the model’s learned distribution (a “high‑probability tool caller”) overrides the soft constraint of the prompt.

Prompt is a soft constraint; data distribution is a hard constraint.

Therefore, merely tightening the prompt is ineffective in real‑world traffic.

First brake: systematically add “non‑tool” samples

Insert a proportion of training examples that do not involve tool calls. These should be diverse, semantic, and clearly outside the tool‑calling scope.

Pure chat samples

“Did you have lunch?”

“What does traveling mean to you?”

“Is traveling alone lonely?”

These have meaning and emotion but no tool‑execution requirement.

Illegal / over‑privileged requests

“Check neighbor Wang’s call records.”

“Locate a person’s current whereabouts.”

The model should learn to refuse such requests rather than invoke a tool.

Key proportion

Non‑tool samples should occupy 10%–15% of the dataset.

Too few samples fail to curb the rush; too many make the model reluctant to call tools.

Second brake: force the model to “think before acting”

Replace the simple User → Tool Call format with an explicit Thought chain: User → Thought → Tool Call / Text Reply The Thought step is not shown to the user; it lets the model:

Judge the user’s intent.

Check whether all required parameters are present.

Decide if a tool call is truly needed.

Example:

Thought:
The user wants the weather but did not provide a date, so we should ask for it first.

If the model skips the Thought step or directly outputs tool_calls, a high loss penalty is applied during training, encouraging the model to pause.

“Should I think for a second first?”

Third brake: negative sampling

Construct samples that look like tool requests but semantically do not require a tool. Label them as plain text replies and explicitly tell the model that no JSON should be produced.

What “looks like a request” means

Example: “I checked Beijing’s weather yesterday, it was nice.” The sentence contains the location and keyword “weather” but is a statement, not a request. If the model still calls get_weather, it is a false positive.

How we build negative samples

Include tool‑related keywords.

Include parameter clues.

Ensure the overall semantics do not require a tool.

Mark the output type as ordinary text.

Explicitly tell the model “no JSON here”.

After repeated exposure, the model learns that not every “request‑like” utterance should trigger a tool.

Why this matters in interviews

Understanding the root cause, the negative‑sampling design, the proper non‑tool sample ratio, and the Thought‑chain constraint distinguishes candidates who can ship reliable agents from those who only know prompt tricks.

“This person not only knows how to fine‑tune a model but also considers the system’s behavior after deployment.”

A well‑balanced agent is not a “tool‑crazy monster” but an executor with judgment, restraint, and boundary awareness.

From demo to production, the key is to give the model a brake system instead of removing the accelerator.