Tool Use

Analogy - The Smart Assistant

Imagine a brilliant intern who knows everything from textbooks but has never used a computer, phone, or calculator. They can discuss quantum physics but cannot check today's weather. Function calling is like giving that intern a phone, a calculator, and access to the internet - now they can actually DO things, not just TALK about things.

The Core Idea:

By default, LLMs can only generate text. They cannot check live stock prices, send emails, query databases, or book tickets. Tool use (also called function calling) lets LLMs request the execution of external functions. The LLM does not run the function itself - it tells YOUR application which function to call and with what arguments. Your code executes the function and sends results back to the LLM.

The Flow:

1. 1. You define tools: Tell the LLM what functions are available (name, description, parameters)
2. 2. User asks something: "What is the weather in Mumbai?"
3. 3. LLM decides to use a tool: Instead of guessing, it outputs a structured function call: get_weather(city="Mumbai")
4. 4. Your app executes it: Calls the real weather API, gets result: 32 degrees C, Humid
5. 5. LLM gets the result: Formulates a natural language response using real data

Tool Definition - JSON Schema:

You describe your tools using JSON Schema. Each tool has a name, description (crucial for the LLM to decide when to use it), and parameter definitions with types and descriptions.

Tool: get_stock_price
Description: "Get the current stock price for an Indian company"
Parameters:
  - symbol (string, required): "NSE stock ticker, e.g. RELIANCE, TCS"
  - exchange (string, optional): "NSE or BSE, default NSE"

LLM Decision Making:

When a user message comes in, the LLM has three choices:

• Respond directly: If it can answer from its training data ("What is HDFC Bank?")
• Call one tool: If it needs one piece of external data ("HDFC Bank share price?")
• Call multiple tools: If it needs several pieces of data ("Compare HDFC and ICICI prices") - parallel or sequential calls

Why Descriptions Matter:

The LLM picks the right tool based on the description you provide. A vague description like "gets data" will confuse the model. A good description like "Get the current real-time stock price for any company listed on NSE/BSE. Returns price in INR, daily change, and volume" helps the LLM make the right choice every time.

Key Insight:

The LLM does NOT execute the function. It only generates a structured JSON saying "I want to call function X with parameters Y." YOUR application code is responsible for actually running the function, handling errors, and sending results back. This is a safety feature - you control what actually gets executed.

OpenAI (GPT-4, GPT-4o):

Uses the term "function calling" or "tools". You pass a "tools" array in the API request. The model responds with "tool_calls" containing function name and arguments as JSON. Supports parallel tool calls (multiple functions at once). Introduced "strict mode" for guaranteed valid JSON output.

Anthropic (Claude):

Uses the term "tool use". You define tools with input_schema (JSON Schema). Claude responds with "tool_use" content blocks. Claude is known for being very good at deciding WHEN to use tools vs when to answer directly. Also supports a special "computer use" tool for GUI interaction.

Google (Gemini):

Uses "function calling" terminology. Similar pattern - define function declarations, Gemini responds with function call parts. Supports automatic function calling mode where the SDK can execute functions for you.

Key Differences:

Mistake 1: Too Many Tools

Giving an LLM 50+ tools confuses it. It struggles to pick the right one. Keep your tool set focused. If you have many tools, use a two-stage approach: first LLM call picks the category, second call uses tools from that category only.

Mistake 2: Poor Tool Descriptions

Vague descriptions like "process data" or "helper function" cause wrong tool selection. Write descriptions as if explaining to a smart human who has never seen your codebase. Include what it does, when to use it, and what it returns.

Mistake 3: No Error Handling

Tools fail - APIs timeout, databases go down, input validation fails. Always send error messages back to the LLM in a structured way so it can either retry, try a different approach, or inform the user gracefully.

Best Practices:

• Validate inputs: Check LLM-generated parameters before executing. The LLM might hallucinate invalid values.
• Set timeouts: Tool execution should have reasonable timeouts. Do not let a slow API hang your agent.
• Log everything: Log every tool call, its parameters, and results. Essential for debugging.
• Limit scope: Do not give tools write access unless absolutely needed. Read-only tools are safer.
• Human-in-the-loop: For dangerous operations (delete, transfer money), require human approval before executing.

Pattern 1: Information Retrieval

Most common pattern. LLM fetches real-time data that it does not have in training data.

• Weather lookup, stock prices, cricket scores
• Database queries for customer information
• Document search in company knowledge base

Pattern 2: Action Execution

LLM takes actions in the real world on behalf of the user.

• Send email, create calendar event, post to Slack
• Book a flight, order food, schedule a cab
• Create Jira ticket, deploy code, run tests

Pattern 3: Multi-Step Workflows

LLM chains multiple tool calls to accomplish complex tasks.

• Step 1: Search flights Delhi to Goa
• Step 2: Check user's calendar for conflicts
• Step 3: Book the best available flight
• Step 4: Send confirmation email to user

Pattern 4: Computation Offloading

LLMs are bad at precise math and data processing. Offload these to tools.

• Complex calculations (EMI, tax, compound interest)
• Data analysis on large CSV files
• Code execution for precise results

Q: What is function calling in LLMs and why is it important?

Q: Does the LLM actually execute the function? Explain the safety implications.

Q: How do you handle errors when a tool call fails?