Dynamic Tool Selection Unpacked: Let the Agent Choose the Right Tool with Three Strategies

In the previous post we dissected five intent‑recognition designs; this article goes one step further to explain how an agent decides which tool to invoke after intent is identified.

Binding every available tool (e.g., 50 tools) to the LLM leads to three major problems: token consumption triples, the tool‑selection error rate exceeds 30 %, and occasional “hallucinated” tool calls appear. A real‑world benchmark compares three approaches:

Static full binding (50 tools): ~8000 tokens per call, 28 % error, 3.2 s latency.

Dynamic selection via semantic retrieval (reduce to 3 tools): ~1200 tokens, 6 % error, 1.1 s latency.

Dynamic selection via LLM routing (reduce to 5 tools): ~2400 tokens, 4 % error, 1.8 s latency.

Token consumption drops six‑fold and error rates improve four‑to‑five times. The root causes are context‑window pollution (each tool description adds 100‑300 tokens), interference between similar tools, and the concrete cost of tokens (e.g., GPT‑4o at $5 per million input tokens).

Three dynamic‑tool‑selection strategies form a cost‑speed‑accuracy triangle:

Strategy 1: Vector semantic retrieval (cheapest)

Tool descriptions are embedded; at request time the user query vector is compared to all tool vectors and the top‑K tools are passed to the LLM. Advantages: pure local computation, latency < 50 ms, no extra LLM call. Drawback: quality depends on description accuracy.

Strategy 2: LLM second‑stage routing (most accurate)

A lightweight LLM (e.g., GPT‑4o‑mini) first predicts a list of tool IDs, which are then supplied to the main LLM. Advantages: highest accuracy; drawback: an extra LLM call adds 200‑500 ms latency and cost.

Strategy 3: Rule + semantic hybrid (most robust)

Hard rules/keywords force inclusion of certain tools, then semantic retrieval fills the remaining slots. This yields zero errors on high‑frequency scenarios while still handling ambiguous queries.

Choosing a strategy depends on tool count and complexity:

≤ 20 clear tools → Strategy 1.

20‑100 tools with complex business logic → Strategy 3.

> 100 tools or fuzzy semantics → Strategy 2.

Implementation of Strategy 1 with LangGraph (TypeScript)

import { tool } from "@langchain/core/tools";
import { OpenAIEmbeddings } from "@langchain/openai";
import { z } from "zod";

// Define tools
const allTools = [
  tool(async ({ city }) => `${city} today sunny, 26°C, SE wind 3`, {
    name: "weather_query",
    description: "Query real‑time weather for a city",
    schema: z.object({ city: z.string().describe("city name") })
  }),
  tool(async ({ query }) => `Search results: ${query} ...`, {
    name: "web_search",
    description: "Search the web for latest info",
    schema: z.object({ query: z.string().describe("search keyword") })
  }),
  // ... other tools omitted for brevity
];

// Build vector index (run once at startup)
const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" });
const toolDescriptions = allTools.map(t => `${t.name}: ${t.description}`);
const toolVectors = await embeddings.embedDocuments(toolDescriptions);
const toolRegistry = allTools.map((t, i) => ({ tool: t, vector: toolVectors[i] }));

function cosineSimilarity(a, b) {
  const dot = a.reduce((s, v, i) => s + v * b[i], 0);
  const normA = Math.sqrt(a.reduce((s, v) => s + v * v, 0));
  const normB = Math.sqrt(b.reduce((s, v) => s + v * v, 0));
  return dot / (normA * normB);
}

async function selectTools(query, topK = 3) {
  const queryVector = await embeddings.embedQuery(query);
  const scored = toolRegistry.map(({ tool, vector }) => ({ tool, score: cosineSimilarity(queryVector, vector) }));
  scored.sort((a, b) => b.score - a.score);
  return scored.slice(0, topK).map(s => s.tool);
}

// LangGraph integration (simplified)
const graph = new StateGraph(AgentState)
  .addNode("select_tools", async state => {
    const last = state.messages[state.messages.length - 1];
    const tools = await selectTools(last.content as string);
    return { selectedTools: tools };
  })
  .addNode("agent", async state => {
    const { messages, selectedTools } = state;
    const llmWithTools = llm.bindTools(selectedTools);
    const response = await llmWithTools.invoke(messages);
    return { messages: [response] };
  })
  .addEdge("__start__", "select_tools")
  .addEdge("select_tools", "agent")
  .addConditionalEdges("agent", shouldContinue, { tools: "tools", END: END })
  .addEdge("tools", "select_tools")
  .compile();

The graph ensures that each conversation round runs select_tools before the LLM, guaranteeing up‑to‑date tool selection.

Hybrid rule + semantic selection (Strategy 3) forces tools based on keywords and then supplements with vector retrieval. Example code produces the final tool list ["currency_exchange", "stock_price", "calculator"] for the query “convert 100 USD to CNY and buy some Apple stock”.

Tool description engineering is a hidden variable that dramatically affects retrieval accuracy. Good descriptions enumerate applicable scenarios, trigger keywords, and parameter semantics, while poor descriptions lead to missed matches.

Multi‑turn dialogue requires re‑running the selection step each turn; the graph edge tools → select_tools → agent implements this, adding < 50 ms per turn but preventing stale tool usage.

Common pitfalls (and fixes):

When a tool fails, the model may fall back to a lower‑rank tool—wrap tool execution in explicit error handling and return a standard error message.

Setting Top‑K too low hides the correct tool—start with K = 5 and adjust based on logs.

Tool vectors are static—re‑build the index whenever the tool registry changes.

Similar‑keyword tools can be confused—add “reverse‑exclusion” clauses in descriptions to clarify scope.

In summary, static full binding is a production anti‑pattern; vector retrieval offers the cheapest baseline; hybrid rule + semantic selection provides the most robust coverage; high‑quality tool descriptions are essential; and re‑selecting tools each turn prevents stale selections. The next article will cover failure handling, retries, fallbacks, and human hand‑off strategies.