Skip to content
Underread
Guides

Why the Model Ignores Your Tool, and How to Make It Call One

It answers from memory instead of calling your function. Nearly always the description is the problem, not the model.

By Vemuri Abhiram6 min read
Close-up of a circuit board with fine traces
Close-up of a circuit board with fine traces
On this page

You've defined a tool, wired it up, and the model ignores it. It answers the question from its own knowledge instead, confidently and often wrongly:

"The current order status is likely to be 'shipped'."

No tool call in the response. No error anywhere. The function you wrote is never invoked, and nothing in the API tells you why.

#The 5 causes, in order

# Cause Tell Time
1 You're reading the wrong field Text is empty or odd 5 min
2 Description too vague Works sometimes 20 min
3 Model thinks it already knows Confident wrong answers 20 min
4 Too many tools to choose from Picks the wrong one 1 hour
5 Not forced when it should be Inconsistent 10 min

#1. Make sure it isn't calling it already

Before changing anything, print the whole response object. A tool call doesn't arrive as text — it's a separate structure, and if you're reading only the message content you'll see nothing and conclude the model ignored you.

print(resp.model_dump_json(indent=2))

Check finish_reason as well. A response that stopped because it wants to call a tool reports that explicitly, and seeing it tells you the model did exactly what you wanted and your code didn't notice. That's a happier bug than the alternative.

#2. Write the description for a new colleague

This is the fix that resolves most cases, and it's unglamorous. The model chooses tools almost entirely from their descriptions, and most descriptions are written as if the reader already knows what the tool is for.

Compare these 2:

"Gets order data."

"Look up the current status, items and delivery estimate for a
customer order by its order id. Use this whenever the user asks
about an order, including 'where is my order' and 'has it shipped'.
Returns null if the order id does not exist."

The second one tells the model when to reach for it, which is the part the first one leaves entirely to guesswork. Describe the parameters the same way, because people skip them — an order_id described as "the order id, e.g. ORD-10294" is far more useful than one described as "the id".

#3. Tell it not to answer from memory

A model trained on vast amounts of text has a strong prior toward answering. If a question looks answerable, it will often answer rather than call anything. That gets worse for questions that sound general, even when the answer must come from your system.

A line in the system prompt fixes most of it:

Never answer questions about orders, inventory or pricing from your own
knowledge. That information changes constantly and you do not have it.
Always call the appropriate tool. If no tool fits, say so.

The "say so" clause matters more than it looks. Without a permitted way to decline, a model that can't find a suitable tool will often invent an answer, because answering is what it's for.

#4. Fewer tools, better named

Tool selection degrades as the list grows, especially when several tools overlap. If you have get_order, get_order_status and lookup_order_details, the model has to guess, and so would a person.

Consolidate into fewer tools with clearer boundaries. Or split the work, so each request only sees the tools relevant to it. Routing to a small set first, then calling with that subset, consistently works better than offering 30 tools and hoping.

#5. Force the call when it isn't optional

Where a tool must be used, say so in the API rather than asking in the prompt. Every major provider supports requiring a tool call, and it removes the question entirely:

resp = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=messages,
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "get_order"}},
)

Use it when the step is deterministic. A lookup that has to happen before anything else can be answered. Don't use it for a general assistant, where forcing a call on every turn means calling a tool to answer "thanks, that's helpful".

The related pattern is worth knowing: structured output enforces the shape of a response, and tool calling enforces the shape of an action. If what you actually want is a parsed object rather than a side effect, the structured output guide is the better route.

#6. Rewrite the description first, always

Before touching the model, the prompt or the temperature, reread the description.

I have fixed this maybe a dozen times now, and it was the description in at least 9 of them. Not the model. Not the framework. A sentence that assumed the reader already knew what the tool was for.

The test I use is simple. Hand the description to somebody who has never seen the system and ask when they would call it. If they hesitate, the model will too, because it has strictly less context than they do.

Then check the parameter descriptions, which people skip entirely. A parameter described as the id tells the model nothing about format, and a model that is unsure what to pass will often decline to call rather than guess.

#Match your symptom to the fix

What you're seeing Likely cause Go to
Confident answer, no call Answering from memory 2 and 3
Calls sometimes, not always Vague description 2
Calls the wrong tool Too many, or overlapping 4
Empty text, nothing happens Reading the wrong field 1
Needs to call every time Not forced 5

#When it still won't call

Test the tool on its own. One tool, one unambiguous question, no system prompt, nothing else in the conversation. If it calls reliably there and not in your real setup, something in that setup is the cause. Usually one of 2 things. An instruction in the system prompt competing with the tool description, or a long history where earlier turns answered similar questions directly and set a pattern.

If it won't call even in isolation, the description is still the problem. Read it as though you've never seen the system and ask whether you'd know when to use this. If the answer involves any inference about what the function probably does, rewrite it until it doesn't.

Frequently asked questions

Is this different between providers?

The mechanism is the same and the parameter names differ. All the major APIs let you supply tools, read back a structured call, and require one. A description that works well on one model generally works well on another.

Should I always force a tool call?

No. Forcing is right for a deterministic step that must happen. Forcing on every turn makes a conversational assistant call a function to say hello.

How many tools is too many?

There's no hard number, but selection quality drops as the list grows and as tools overlap. If 2 tools could plausibly answer the same question, that's a problem at any list size.

Can the model call several tools at once?

Most current models can return multiple calls in one response. Handle a list rather than assuming one, or you'll silently drop calls when it does.

Written by

Vemuri Abhiram

Founder & Editor

Vemuri Abhiram is the founder and editor of Underread, where he writes about AI tools, software, and practical technology. He covers hands-on tool reviews, detailed guides, product comparisons, and emerging technologies.

More from Underread, once a week.

The guides worth reading and the tests worth trusting. One email, every Friday.

No spam. Unsubscribe in one click.