Tool Use with Large Language Models: Function Calling and External APIs Explained

Large language models (LLMs) can’t read the news, check your bank balance, or book a flight on their own. They don’t have live access to the world outside their training data. But that’s changing. With function calling, LLMs can now ask for help - and get it - without breaking the conversation. This isn’t magic. It’s a structured way for models to talk to real tools and APIs, turning them from static text predictors into active problem solvers.

How Function Calling Actually Works

Function calling lets an LLM decide when to use an external tool. Instead of guessing the current stock price or the weather in Tokyo, it generates a clean JSON request like this:

{
  "name": "get_weather",
  "arguments": "{\"city\": \"Tokyo\", \"unit\": \"celsius\"}"
}

The model doesn’t run the code. It just tells your system: “Here’s what to do.” Your app then calls the weather API, gets the answer, and feeds it back to the model. The LLM then summarizes the result in plain language - like a helpful assistant who just looked something up for you.

This process has four clear steps:

1. Analysis: The model reads your question and decides if it needs a tool.
2. Selection: It picks the right function from a list you provided.
3. Execution: Your system runs the function with the extracted parameters.
4. Synthesis: The model uses the result to give a natural-language answer.

It’s not just about retrieving data. You can use it to calculate mortgage payments, pull customer records from a database, or even trigger a workflow in your CRM. The key is that the model stays in control of the conversation - it just gets smarter by reaching outside itself.

Major Models and How They Compare

Not all LLMs handle function calling the same way. Here’s how the big players stack up as of late 2025:

OpenAI’s GPT-4 Turbo leads in tool availability. If you want to plug into 2,400+ pre-built tools - from Stripe to Slack - it’s your best bet. But it’s also the strictest. If you miss a single required field in the JSON, the call fails. No warnings. No guesses.

Claude 3.5 Sonnet is the opposite. It’s more forgiving. If you say “What’s the weather like today?” without specifying a city, Claude will often infer it from context - like your last message or user profile. That’s why it scores higher on ambiguous inputs.

Gemini 1.5 Pro shines when you need to chain tools together. Imagine asking: “Find my last order from Amazon, check the delivery status, and email me the tracking number.” Gemini handles that flow better than anyone else. But it’s slower - 27% longer response times due to its multi-turn refinement.

Qwen3 is the quiet innovator. Before calling any tool, it writes out its reasoning: “I need to check the weather because the user asked about outdoor plans. I’ll use the get_weather function with city=Tokyo.” That transparency builds trust, especially in enterprise settings where users want to understand why the AI did what it did.

Why Function Calling Matters

Before function calling, LLMs were prone to hallucinations - especially with time-sensitive or personal data. Ask a model when your last invoice was paid, and it might make up a date. Ask about today’s stock price, and it’ll give you data from 2023.

With function calling, those errors drop by up to 68%, according to UC Berkeley’s Pieter Abbeel. Why? Because the model isn’t guessing anymore. It’s using real data.

Businesses are seeing real results:

• Customer service bots resolve issues 53% faster by pulling data directly from support systems.
• Financial assistants give accurate loan estimates by connecting to internal calculators.
• HR chatbots answer policy questions by accessing internal databases instead of relying on outdated training.

Gartner reports that 78% of enterprises now use function calling in their LLM apps - up from just 28% in mid-2024. The global market for this tech hit $2.4 billion in 2025 and is expected to grow to nearly $10 billion by 2028.

What You Need to Build It

Getting started isn’t hard, but it’s not plug-and-play either. Here’s what you actually need:

1. A function schema: Define each tool with a name, description, and parameters in JSON Schema format. Example:

{
  "name": "get_user_email",
  "description": "Fetch the email address of a user by their ID",
  "parameters": {
    "type": "object",
    "properties": {
      "user_id": {
        "type": "string",
        "description": "The unique ID of the user"
      }
    },
    "required": ["user_id"]
  }
}

2. A function router: Your code needs to listen for the model’s JSON output and execute the matching function. This is usually a switch-case or dictionary lookup in your backend.
3. Error handling: What if the API is down? What if the user ID doesn’t exist? You need fallback responses. Most developers spend over half their debugging time here.
4. Conversation flow: Design how the model will respond after getting a result. Should it say, “I found your email: [email protected]”? Or “I couldn’t find your account - can you confirm your ID?”

Experts recommend using four-shot prompting: show the model four clear examples of how to use the tool before asking it to act. This boosts accuracy more than any other technique.

Common Pitfalls and How to Avoid Them

Most teams hit roadblocks. Here are the top three issues - and how to fix them:

1. Parameter Validation Errors

OpenAI’s system demands perfect JSON. If you pass a string where it expects a number, or forget a required field, the call fails silently. Developers report spending 53% of their debugging time on this alone.

Solution: Use a schema validator in your dev environment. Test every parameter type. Add logging to capture failed calls. Don’t assume the model will get it right.

2. Ambiguous Requests

“Show me my orders.” Which user? Which time period? Which system? Models often pick the wrong context.

Solution: Build clarification loops. If the model can’t be sure, it should ask: “Which account would you like me to check?” Don’t force it to guess.

3. Infinite Loops

What if the model keeps calling the same tool over and over? “What’s the weather?” → “I checked the weather.” → “What’s the weather?” → …

Solution: Set a max_turns limit. Most platforms allow you to cap function calls at 5-10 steps. That’s usually enough.

Real-World Limits

Function calling isn’t a cure-all. It won’t help if you don’t have the right tools.

A Johns Hopkins study found that without a medical diagnostic API, LLMs failed 41% of the time on basic diagnosis questions. The model could describe symptoms, but couldn’t interpret lab results or access patient history.

And there’s a hidden risk: parameter injection. If a user types “Show me all orders from user_id=123; DROP TABLE users;”, and your system doesn’t sanitize input, you’ve opened a security hole. Dr. Percy Liang from Stanford found 37% of early implementations were vulnerable to this.

Always validate and sanitize inputs. Treat function parameters like user form data - never trust them.

What’s Next?

The field is moving fast. In October 2025, OpenAI launched GPT-5 with “adaptive parameter validation” - it now suggests corrections if a parameter is wrong. Claude 3.5 added “tool chaining,” letting models automatically sequence multiple calls without human input.

Google’s “tool grounding” feature, released in December 2025, cross-checks function results against the model’s own knowledge. If the weather API says it’s 80°F but the model knows it’s snowing in that city, it flags the conflict.

By 2027, Forrester predicts 92% of enterprise LLM apps will use function calling. The big question isn’t whether you should use it - it’s how well you’ll build it.

The future belongs to models that don’t just talk - they act. And function calling is how they do it.

Next Steps

If you’re ready to try function calling:

1. Start with one simple tool - like a weather API or a calculator.
2. Define the function schema exactly as the API expects.
3. Build a basic router that prints the model’s JSON output before calling anything.
4. Test with 10-20 real user questions. Watch where it fails.
5. Use four-shot prompting to improve accuracy.
6. Log every failed call. Fix the most common errors first.

Don’t try to automate everything at once. Start small. Get the basics right. Then scale.