Structured outputs & tool calling

Free-form text is great for chat, but your code needs data: a category, an amount, a date. Modern LLM APIs offer two features for this: structured outputs (the model returns JSON matching a schema) and tool calling (the model asks your code to run a function and uses the result).

Structured outputs with a schema

Describe the shape you want as JSON Schema (or a Pydantic model), send it with the request, then validate the reply. The example uses the OpenAI-compatible Python SDK style, which many providers and local servers support; check your provider's docs because parameter names and model names change over time.

from openai import OpenAI
from pydantic import BaseModel, ValidationError

class Expense(BaseModel):
    vendor: str
    amount_inr: float
    gst_percent: float | None
    category: str

client = OpenAI(base_url="https://api.example-llm.com/v1", api_key="...")  # any compatible endpoint
resp = client.chat.completions.create(
    model="your-model-name",
    messages=[
        {"role": "system", "content": "Extract the expense. Reply only with JSON."},
        {"role": "user", "content": "Paid Sharma Stationers Rs 1,180 incl 18% GST for printer paper"},
    ],
    response_format={"type": "json_schema", "json_schema": {
        "name": "expense", "schema": Expense.model_json_schema()}},
    temperature=0,
)
try:
    expense = Expense.model_validate_json(resp.choices[0].message.content)
except ValidationError as err:
    expense = None      # retry once with the error message, then fall back to a human

Even with schema enforcement, validate business rules yourself (amount > 0, category in your allowed list).

Tool (function) calling

You describe tools with a name, description and JSON Schema for arguments. The model does not execute anything; it returns a request like {"name": "get_order_status", "arguments": "{...}"}. Your code runs the function and sends the result back in a follow-up message, and the model then writes the final answer.

import json

TOOLS = [{"type": "function", "function": {
    "name": "get_order_status",
    "description": "Look up a customer's order by order ID",
    "parameters": {"type": "object",
                   "properties": {"order_id": {"type": "string"}},
                   "required": ["order_id"]},
}}]

def get_order_status(order_id: str) -> dict:
    return {"order_id": order_id, "status": "Shipped", "city": "Kolkata"}   # really: a DB query

messages = [{"role": "user", "content": "Where is my order A-1042?"}]
first = client.chat.completions.create(model="your-model-name", messages=messages, tools=TOOLS)
msg = first.choices[0].message
if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_order_status(**args) if call.function.name == "get_order_status" else {"error": "unknown tool"}
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    final = client.chat.completions.create(model="your-model-name", messages=messages, tools=TOOLS)
    print(final.choices[0].message.content)

The same idea over plain HTTP

Under the hood this is just a POST with JSON; the request body contains model, messages, tools and the response contains choices[0].message.tool_calls. Any language with an HTTP client can do it.

Common mistakes

• Trusting tool arguments blindly. Check the user is allowed to see order A-1042 before querying.
• Letting the model call dangerous tools (refunds, deletes) without confirmation.
• Vague tool descriptions; the model chooses tools based on them.
• Not handling the case where the model answers directly instead of calling a tool.