Why Does My AI Agent Choose the Wrong Tool? A Practical Diagnosis

When an agent chooses the wrong tool, the model is the most visible part of the failure. It is not always the cause.

The wrong tool may not have been available at all. Two tools may describe the same job. The model may have selected the right tool but supplied the wrong argument, or the tool may have returned an ambiguous result that pushed the next decision off course. Treat “wrong tool” as the symptom. The first job is to locate the decision boundary.

Why does an AI agent choose the wrong tool?

An AI agent chooses the wrong tool when the tool surface gives it a plausible but incorrect path for the user’s request. The usual causes are an over-large candidate set, overlapping descriptions, weak input distinctions, an unconstrained runtime, or a result that does not make the next valid action clear.

In a tool-calling flow, your application sends the model a prompt and a list of tools it may consider. The model returns a tool call, your application executes it, and the result goes back into the conversation for the next decision (OpenAI’s function-calling guide). The model is choosing from the interface you supplied, not from the complete set of operations your backend happens to support.

That interface normally contains a tool name, a description, and an input schema. MCP defines the same basic shape and also allows an output schema and behavior annotations (MCP Tools specification). Those fields are not documentation for humans alone. They are part of the information available at the choice point.

The first diagnostic question is therefore not “how do I prompt it harder?” It is: what choices did the model actually see, and what made the chosen one look reasonable?

Is it actually the wrong tool?

Separate these failures before you change anything:

What the trace shows	What may really be wrong	First check
A search tool is called instead of a record lookup	The descriptions overlap or the exact identifier is not visible	Compare the competing descriptions and the user’s available identifiers
The intended tool is called with an invalid argument	Tool choice was correct; the input contract is weak	Validate the arguments against the schema
The intended tool is called for the wrong customer or document	Entity binding failed after selection	Check how the target ID was resolved and revalidated
The tool returns “done” but the task is not done	Execution or result semantics are misleading	Inspect the tool’s output status and postcondition
A tool that should be used is never called	It was hidden, filtered, deferred, or unavailable	Log the callable tool set for that turn

This distinction saves time. A stricter schema can reject an invalid customer_id, but it cannot prove that get_customer was the right tool instead of search_customers. Conversely, a perfect description cannot repair a runtime that accidentally hides the only relevant tool.

What should you inspect first? Use the WRONG framework

The following is an original diagnostic synthesis for this article. Use it in order. Each gate removes one class of explanation before you rewrite the prompt.

Gate	Question	Evidence	Typical repair
W: Was it available?	Could the model call this tool in this turn?	Request payload, allowed subset, deferred-tool events, conditional enablement, permissions	Fix loading or routing before changing descriptions
R: Rival tools?	Which other tools looked plausible for the same request?	Names, descriptions, examples, aliases, negative boundaries	Make the tools distinguishable or consolidate them
O: Operation schema?	Does the schema make the intended operation and inputs explicit?	Required fields, enums, identifiers, read/write shape, validation errors	Encode distinctions in the schema and application code
N: Next evidence?	Did the result tell the model what happened and what it may do next?	Status, error code, target ID, state version, next action	Return concise, structured, high-signal results
G: Guard the consequence?	What stops a plausible but harmful choice from becoming an external effect?	Allowlist, authorization, dry-run, approval, idempotency, executor recheck	Add a deterministic policy boundary

W: Was the tool available?

Capture the exact candidate set for the failing turn. “The agent has access to the CRM” is not enough. The relevant question is whether get_customer_record was in the model-facing request after feature flags, permissions, conditional filters, namespace loading, and tool search had run.

This matters in systems that load tools on demand. OpenAI documents tool search as a way to defer large tool surfaces and load a relevant subset at runtime. Its Agents SDK also supports conditionally enabled tools that are hidden from the model when disabled (OpenAI function calling, OpenAI Agents SDK tools). A missing candidate cannot win a selection test.

Add these fields to the trace for one failing run:

run_id
turn_id
available_tools: [name, version, permission]
deferred_tools_loaded: [name]
allowed_tools: [name]
selected_tool
selected_arguments

If the intended tool is absent, repair exposure first. Do not use a prompt edit to compensate for a broken tool loader.

R: Which rival tools are plausible?

Put the competing definitions side by side. A tool called customer_search, described as “find customer information,” and a tool called get_customer, described as “retrieve customer information,” give the model little help when the user says “look up this customer.” Both names and descriptions point toward the same action.

Descriptions need boundaries, not just benefits. Anthropic’s current tool guidance asks authors to explain what a tool does, when it should and should not be used, what its parameters mean, and its limitations. It also recommends consolidating related operations and using meaningful namespacing as the library grows (Anthropic’s tool-definition guidance). OpenAI likewise recommends clear function names, instructions, parameter descriptions, and useful edge cases (OpenAI’s function-calling guide).

For every pair of rivals, write one sentence that separates them:

lookup_order: use when the user gives an order number and needs the current record.
search_orders: use when the user does not have an order number and needs candidates.
Do not use search_orders when an exact order number is already available.

The negative sentence is not magic. It is a test of whether the boundary exists. If you cannot write a clean “use this, not that” rule, the tools may be too similar to expose as separate choices.

O: Is the operation schema doing useful work?

A schema should make invalid states difficult to express. Use enums for finite choices, require identifiers that the application does not already know, and keep read operations visibly different from writes. OpenAI recommends using enums and object structure to represent valid states and offloading known values to code instead of asking the model to fill them in (OpenAI’s function-calling guide).

Strict calling helps with one part of the problem. OpenAI documents strict mode as a way to make generated arguments adhere to the function schema rather than remain best effort (OpenAI strict mode guidance). That is valuable, but it is not a semantic judge. A perfectly valid call to the wrong function is still a wrong tool.

Use the schema to express the choice where possible:

• search_orders accepts a free-text query and returns candidates.
• get_order accepts one known order ID and returns one record.
• cancel_order accepts one confirmed order ID plus a reason and has a write boundary.

Do not expose cancel_order(order_id, customer_name, maybe_reason, maybe_force) when the runtime already knows the order ID and the policy already decides whether force is allowed. Pass known values in application code and leave the model fewer opportunities to invent or confuse them.

N: What did the tool result make possible?

The first call can be reasonable and the next choice can still be wrong. A result such as “request accepted” does not tell the agent whether the action completed, entered a queue, failed validation, or changed nothing.

Return the smallest high-signal result that supports the next decision. Anthropic recommends stable identifiers and concise results, and MCP supports structured tool content plus an explicit error indicator (Anthropic’s tool-definition guidance, MCP Tools specification). A useful provider-neutral result might look like this:

{
  "status": "success | pending | retryable_error | permanent_error",
  "code": "ORDER_FOUND",
  "entity_id": "ord_123",
  "state_version": "v18",
  "changed": false,
  "next_allowed_action": "get_order | cancel_order | ask_user | none",
  "safe_to_repeat": true
}

This is an illustrative contract, not a vendor schema. The point is to remove guesswork from the next choice. If a write is queued, return pending and a durable reference. If a request is permanently invalid, return a permanent error rather than inviting a new cosmetic variation of the same call.

G: How is the consequence guarded?

Tool selection is a model decision; authorization is an application responsibility. Before an external effect, check the selected tool, normalized arguments, actor, permissions, target, and current state again in code.

This matters even with MCP. The specification says annotations describing tool behavior must be treated as untrusted unless they come from a trusted server (MCP Tools specification). A label such as “read-only” should not be your only protection against a write.

For a consequential tool, add at least one of these boundaries: a read-only or dry-run mode, an allowlist of callable tools, an idempotency key, a human approval step, or an executor-side recheck. The control should fail closed when the selected tool is outside policy or the target has changed.

How do you fix the wrong choice?

Repair the narrowest layer that explains the trace.

1. Freeze the failure. Save the prompt, model and dated endpoint if known, exact candidate set, selected name, normalized arguments, tool result, permissions, and final effect. Redact secrets. If you cannot reproduce it, keep the trace and label the cause uncertain.
2. Shrink the choice. Remove tools irrelevant to this turn. Use a deterministic workflow, an allowlist, a namespace, or a deferred-loading mechanism when the candidate surface is large. OpenAI’s current guide gives fewer than 20 initially available functions as a soft suggestion and explicitly says to evaluate different counts; treat it as guidance, not a law (OpenAI’s function-calling guide).
3. Rewrite the rival definitions. Give each tool a clear name, purpose, positive use condition, negative boundary, input meaning, output meaning, and important limitation. The OpenAI Agents SDK derives descriptions from docstrings and schemas from function signatures unless you override them, so inspect the generated model-facing definition rather than only the source code (OpenAI Agents SDK tools).
4. Move stable routing into code. If a known identifier, user permission, or workflow stage determines the only valid operation, choose or filter the tool in application code. Let the model handle the ambiguous part, not a decision your program already knows.
5. Constrain the model when the choice is known. OpenAI documents allowed tool subsets and forced tool choice; use the equivalent control in your provider when a stage has one valid tool or a small declared set (OpenAI tool choice). Do not force a tool merely to hide an unresolved ambiguity.
6. Make results discriminating. Return status, stable IDs, error class, state, and next allowed action. Avoid long logs and polite prose that leave the model to infer whether the operation succeeded.
7. Retest the boundary. Keep the original failure as a regression case and add nearby cases that could select the same rival. Check both the selected tool and the external effect.

What should a good tool contract contain?

Use this card when reviewing one tool against its nearest rivals. It is intentionally provider-neutral; map the fields to your SDK rather than sending the card as-is to a model.

name: get_order
purpose: Return the current record for one known order.
use_when:
  - The user or application has an exact order_id.
do_not_use_when:
  - The user has only a name, email, or free-text description.
  - The request asks to change, cancel, or refund the order.
inputs:
  order_id:
    type: string
    required: true
    description: Stable order identifier supplied by the user or a trusted prior lookup.
outputs:
  status: success | permanent_error
  entity_id: string
  state_version: string
  next_allowed_action: cancel_order | ask_user | none
side_effect: none
safe_to_repeat: true
failure_contract:
  - code: ORDER_NOT_FOUND
    next_action: ask_user

The card forces a useful review conversation. What makes this tool different from search_orders? Which values are trusted? What does “success” mean? What can safely happen next? If the answers are vague, the model is being asked to infer architecture from labels.

How can you test tool selection without fooling yourself?

Test the choice at the boundary, not only the final answer. The following seven-case matrix is a reusable test design. It does not claim a pass rate or a benchmark result.

Case	Prompt condition	Expected behavior
Ambiguous intent	“Find the order and tell me if I can cancel it”	Use a lookup path, then stop or request the missing cancellation facts; do not write immediately
Exact intent	“Cancel order ord_123 because it arrived damaged”	Choose the cancellation path only if policy and required fields allow it
Conflicting synonym	Use “look up,” “find,” and “retrieve” for the same known ID	Stable routing when the identifier and operation are unchanged
Missing identifier	Ask for one exact record without an ID	Search or ask for a disambiguating field; never invent an ID
Read/write boundary	Ask for the current address, then ask to change it	Read and write calls stay distinct, with the write guarded
Unavailable tool	Remove the intended tool from the candidate set	Explain the limitation or use a declared fallback; never pretend the tool ran
Wrong entity risk	Two customers have similar names	Require a trusted identifier or clarification before a consequential action

For each case, assert the tool name, normalized arguments, candidate set, authorization decision, side effects, and final response. Run the matrix across the model endpoint and configuration you actually deploy. One successful demo cannot tell you whether the boundary is reliable.

Keep this test focused on selection and routing. If the selected action then repeats without progress, use the AI agent loop guide. If you are turning these cases into a pre-release gate, use the AI agent evaluation guide for the broader release decision.

What usually fails as a fix?

“Just add a stronger system prompt”

A prompt can clarify a real ambiguity. It cannot make an unavailable tool available, repair a misleading schema, or enforce an authorization rule. Change the model-facing contract and the runtime boundary when those are the failing layers.

“Turn on strict mode”

Strict schemas are useful for argument shape. They do not turn tool selection into a deterministic classifier. Keep the selection test separate from the argument-validation test.

“Give the model every tool so it can choose”

More capability can create a harder routing problem. OpenAI recommends evaluating the number of initially available functions and using tool search for large surfaces; Anthropic recommends fewer, more capable tools and meaningful namespacing (OpenAI’s function-calling guide, Anthropic’s tool-definition guidance). If a tool is rare or irrelevant in the current stage, do not make the model compare it on every turn.

“Add a second model to judge the first”

A second model can be useful in a deliberately tested architecture, but it also creates another routing decision, another latency and cost path, and another failure mode. First make the candidate set, contracts, and policy checks legible. A deterministic filter is often easier to inspect than an extra opinion.

“Log only the final answer”

The final answer may sound correct after the wrong tool returned plausible data. Save the candidate set, selected tool, arguments after validation, result status, and verified effect. Without those fields, you are debugging a story instead of a decision.

Should you fix it yourself or ask for help?

Handle it internally when the tool list is small, the failure is reproducible, the side effects are reversible, and one team owns the schemas and executor. Start with the WRONG framework, make one contract change, and run the seven cases before changing several variables at once.

Ask for an architecture or reliability review when the tool surface spans several teams, the wrong choice can send messages or change records, the available set differs by user or tenant, or nobody can reconstruct what the model was allowed to call. The useful review packet is small: one failing trace, the model-facing tool definitions, the permission decision, the tool result, and the expected effect.

Marius’s one-to-one AI consulting page describes the kind of work that can be done on a real workflow. The article is complete without that step. If you ask for help, bring the trace and the contract card rather than a screenshot of the final answer.

The shortest reliable rule is this: make the right tool the clearest available choice, make the wrong tools unavailable or visibly different, and make the executor verify the choice before it can matter.

For the next step, take one real wrong-tool trace and fill in the five WRONG gates. If you cannot answer “what was available?” or “what could safely happen next?”, you have found the repair boundary.