Mastering LLM Function Calling: Theory, Workflow, and Hands‑On Code

Why Function Calling?

Large language models (LLMs) are fundamentally probabilistic token predictors that cannot directly access remote services, execute code, or query live data. This limitation creates two major problems: inability to fetch real‑time information and inability to perform actions in the external environment.

How Function Calling Solves the Problem

Function Calling introduces a structured communication protocol between the LLM and an external application. When the model detects that it cannot answer a query directly, it generates a JSON‑formatted function call request, specifying the function name and required parameters. The application then executes the real‑world operation and feeds the result back to the model, enabling the LLM to produce an informed final response.

Overall Process

Define Tools : Create a JSON Schema describing each callable function (name, description, parameters, required fields).

Intent Recognition & Parameter Extraction : Send the user’s request together with the tool definitions to the LLM. The model decides which tool to call and returns a structured {"name":..., "arguments":...} payload.

Program Execution : The application parses the payload, runs the corresponding local function, and captures the output.

Result Return & Final Reply : The execution result is wrapped in a message with role "tool" and sent back to the LLM, which then generates a natural‑language answer.

Step 1: Define Tools

Example tool definition for a department‑lookup function:

{
  "type": "function",
  "function": {
    "name": "get_department",
    "description": "Retrieve the department of a given employee.",
    "parameters": {
      "type": "object",
      "properties": {
        "name": {"type": "string", "description": "Employee name, e.g., 张三"}
      },
      "required": ["name"]
    }
  }
}

Step 2: Intent Recognition & Parameter Extraction

Send the user query and the tools array to the LLM (using the Volcengine Ark SDK in the example). The model returns a tool_calls object instead of plain text:

{
  "role": "assistant",
  "content": "",
  "tool_calls": [{
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "get_department",
      "arguments": "{\"name\": \"张三\"}"
    }
  }]
}

Step 3: Program Execution

Parse the arguments, invoke the real function, and send the result back as a tool message:

if response_message.tool_calls:
    messages.append(response_message.model_dump())
    for tool_call in response_message.tool_calls:
        if tool_call.function.name == "get_department":
            args = json.loads(tool_call.function.arguments)
            name = args.get("name")
            result = get_department(name)
            messages.append({
                "tool_call_id": tool_call.id,
                "role": "tool",
                "content": result
            })
            second_response = client.chat.completions.create(
                model=model,
                messages=messages
            )
            print("Final reply:", second_response.choices[0].message.content)

Step 4: Return Result & Final Reply

The LLM receives the tool output and produces a natural answer, e.g., "张三所在的部门是研发部。"

Conclusion

Function Calling eliminates the information silo of pure LLMs by granting them access to real‑time data and execution capabilities, turning a text generator into an interactive AI agent capable of querying databases, calling APIs, or running local scripts.