Why Does My AI Agent Choose the Wrong Tool? A Practical Diagnosis

When an agent chooses the wrong tool, the model is the most visible part of the failure. It is not always the cause.

The wrong tool may not have been available at all. Two tools may describe the same job. The model may have selected the right tool but supplied the wrong argument, or the tool may have returned an ambiguous result that pushed the next decision off course. Treat “wrong tool” as the symptom. The first job is to locate the decision boundary.

Why does an AI agent choose the wrong tool?

An AI agent chooses the wrong tool when the tool surface gives it a plausible but incorrect path for the user’s request. The usual causes are an over-large candidate set, overlapping descriptions, weak input distinctions, an unconstrained runtime, or a result that does not make the next valid action clear.

In a tool-calling flow, your application sends the model a prompt and a list of tools it may consider. The model returns a tool call, your application executes it, and the result goes back into the conversation for the next decision (OpenAI’s function-calling guide). The model is choosing from the interface you supplied, not from the complete set of operations your backend happens to support.

That interface normally contains a tool name, a description, and an input schema. MCP defines the same basic shape and also allows an output schema and behavior annotations (MCP Tools specification). Those fields are not documentation for humans alone. They are part of the information available at the choice point.

The first diagnostic question is therefore not “how do I prompt it harder?” It is: what choices did the model actually see, and what made the chosen one look reasonable?

Is it actually the wrong tool?

Separate these failures before you change anything:

What the trace shows	What may really be wrong	First check
A search tool is called instead of a record lookup	The descriptions overlap or the exact identifier is not visible	Compare the competing descriptions and the user’s available identifiers
The intended tool is called with an invalid argument	Tool choice was correct; the input contract is weak	Validate the arguments against the schema
The intended tool is called for the wrong customer or document	Entity binding failed after selection	Check how the target ID was resolved and revalidated
The tool returns “done” but the task is not done	Execution or result semantics are misleading	Inspect the tool’s output status and postcondition
A tool that should be used is never called	It was hidden, filtered, deferred, or unavailable	Log the callable tool set for that turn

This distinction saves time. A stricter schema can reject an invalid customer_id, but it cannot prove that get_customer was the right tool instead of search_customers. Conversely, a perfect description cannot repair a runtime that accidentally hides the only relevant tool.

What should you inspect first? Use the WRONG framework

The following is an original diagnostic synthesis for this article. Use it in order. Each gate removes one class of explanation before you rewrite the prompt.

Gate	Question	Evidence	Typical repair
W: Was it available?	Could the model call this tool in this turn?	Request payload, allowed subset, deferred-tool events, conditional enablement, permissions	Fix loading or routing before changing descriptions
R: Rival tools?	Which other tools looked plausible for the same request?	Names, descriptions, examples, aliases, negative boundaries	Make the tools distinguishable or consolidate them
O: Operation schema?	Does the schema make the intended operation and inputs explicit?	Required fields, enums, identifiers, read/write shape, validation errors	Encode distinctions in the schema and application code
N: Next evidence?	Did the result tell the model what happened and what it may do next?	Status, error code, target ID, state version, next action	Return concise, structured, high-signal results
G: Guard the consequence?	What stops a plausible but harmful choice from becoming an external effect?	Allowlist, authorization, dry-run, approval, idempotency, executor recheck	Add a deterministic policy boundary

W: Was the tool available?

Capture the exact candidate set for the failing turn. “The agent has access to the CRM” is not enough. The relevant question is whether get_customer_record was in the model-facing request after feature flags, permissions, conditional filters, namespace loading, and tool search had run.

This matters in systems that load tools on demand. OpenAI documents tool search as a way to defer large tool surfaces and load a relevant subset at runtime. Its Agents SDK also supports conditionally enabled tools that are hidden from the model when disabled (OpenAI function calling, OpenAI Agents SDK tools). A missing candidate cannot win a selection test.

Add these fields to the trace for one failing run:

run_id
turn_id
available_tools: [name, version, permission]
deferred_tools_loaded: [name]
allowed_tools: [name]
selected_tool
selected_arguments

If the intended tool is absent, repair exposure first. Do not use a prompt edit to compensate for a broken tool loader.

R: Which rival tools are plausible?

Put the competing definitions side by side. A tool called customer_search, described as “find customer information,” and a tool called get_customer, described as “retrieve customer information,” give the model little help when the user says “look up this customer.” Both names and descriptions point toward the same action.

Descriptions need boundaries, not just benefits. Anthropic’s current tool guidance asks authors to explain what a tool does, when it should and should not be used, what its parameters mean, and its limitations. It also recommends consolidating related operations and using meaningful namespacing as the library grows (Anthropic’s tool-definition guidance). OpenAI likewise recommends clear function names, instructions, parameter descriptions, and useful edge cases (OpenAI’s function-calling guide).

For every pair of rivals, write one sentence that separates them:

lookup_order: use when the user gives an order number and needs the current record.
search_orders: use when the user does not have an order number and needs candidates.
Do not use search_orders when an exact order number is already available.

The negative sentence is not magic. It is a test of whether the boundary exists. If you cannot write a clean “use this, not that” rule, the tools may be too similar to expose as separate choices.

O: Is the operation schema doing useful work?

A schema should make invalid states difficult to express. Use enums for finite choices, require identifiers that the application does not already know, and keep read operations visibly different from writes. OpenAI recommends using enums and object structure to represent valid states and offloading known values to code instead of asking the model to fill them in (OpenAI’s function-calling guide).

Strict calling helps with one part of the problem. OpenAI documents strict mode as a way to make generated arguments adhere to the function schema rather than remain best effort (OpenAI strict mode guidance). That is valuable, but it is not a semantic judge. A perfectly valid call to the wrong function is still a wrong tool.

Use the schema to express the choice where possible:

• search_orders accepts a free-text query and returns candidates.
• get_order accepts one known order ID and returns one record.
• cancel_order accepts one confirmed order ID plus a reason and has a write boundary.

Do not expose cancel_order(order_id, customer_name, maybe_reason, maybe_force) when the runtime already knows the order ID and the policy already decides whether force is allowed. Pass known values in application code and leave the model fewer opportunities to invent or confuse them.

N: What did the tool result make possible?

The first call can be reasonable and the next choice can still be wrong. A result such as “request accepted” does not tell the agent whether the action completed, entered a queue, failed validation, or changed nothing.

Return the smallest high-signal result that supports the next decision. Anthropic recommends stable identifiers and concise results, and MCP supports structured tool content plus an explicit error indicator (Anthropic’s tool-definition guidance, MCP Tools specification). A useful provider-neutral result might look like this:

{
  "status": "success | pending | retryable_error | permanent_error",
  "code": "ORDER_FOUND",
  "entity_id": "ord_123",
  "state_version": "v18",
  "changed": false,
  "next_allowed_action": "get_order | cancel_order | ask_user | none",
  "safe_to_repeat": true
}

This is an illustrative contract, not a vendor schema. The point is to remove guesswork from the next choice. If a write is queued, return pending and a durable reference. If a request is permanently invalid, return a permanent error rather than inviting a new cosmetic variation of the same call.

G: How is the consequence guarded?

Tool selection is a model decision; authorization is an application responsibility. Before an external effect, check the selected tool, normalized arguments, actor, permissions, target, and current state again in code.

This matters even with MCP. The specification says annotations describing tool behavior must be treated as untrusted unless they come from a trusted server (MCP Tools specification). A label such as “read-only” should not be your only protection against a write.

For a consequential tool, add at least one of these boundaries: a read-only or dry-run mode, an allowlist of callable tools, an idempotency key, a human approval step, or an executor-side recheck. The control should fail closed when the selected tool is outside policy or the target has changed.

How do you fix the wrong choice?

Repair the narrowest layer that explains the trace.

1. Freeze the failure. Save the prompt, model and dated endpoint if known, exact candidate set, selected name, normalized arguments, tool result, permissions, and final effect. Redact secrets. If you cannot reproduce it, keep the trace and label the cause uncertain.
2. Shrink the choice. Remove tools irrelevant to this turn. Use a deterministic workflow, an allowlist, a namespace, or a deferred-loading mechanism when the candidate surface is large. OpenAI’s current guide gives fewer than 20 initially available functions as a soft suggestion and explicitly says to evaluate different counts; treat it as guidance, not a law (OpenAI’s function-calling guide).
3. Rewrite the rival definitions. Give each tool a clear name, purpose, positive use condition, negative boundary, input meaning, output meaning, and important limitation. The OpenAI Agents SDK derives descriptions from docstrings and schemas from function signatures unless you override them, so inspect the generated model-facing definition rather than only the source code (OpenAI Agents SDK tools).
4. Move stable routing into code. If a known identifier, user permission, or workflow stage determines the only valid operation, choose or filter the tool in application code. Let the model handle the ambiguous part, not a decision your program already knows.
5. Constrain the model when the choice is known. OpenAI documents allowed tool subsets and forced tool choice; use the equivalent control in your provider when a stage has one valid tool or a small declared set (OpenAI tool choice). Do not force a tool merely to hide an unresolved ambiguity.
6. Make results discriminating. Return status, stable IDs, error class, state, and next allowed action. Avoid long logs and polite prose that leave the model to infer whether the operation succeeded.
7. Retest the boundary. Keep the original failure as a regression case and add nearby cases that could select the same rival. Check both the selected tool and the external effect.

What should a good tool contract contain?

Use this card when reviewing one tool against its nearest rivals. It is intentionally provider-neutral; map the fields to your SDK rather than sending the card as-is to a model.

name: get_order
purpose: Return the current record for one known order.
use_when:
  - The user or application has an exact order_id.
do_not_use_when:
  - The user has only a name, email, or free-text description.
  - The request asks to change, cancel, or refund the order.
inputs:
  order_id:
    type: string
    required: true
    description: Stable order identifier supplied by the user or a trusted prior lookup.
outputs:
  status: success | permanent_error
  entity_id: string
  state_version: string
  next_allowed_action: cancel_order | ask_user | none
side_effect: none
safe_to_repeat: true
failure_contract:
  - code: ORDER_NOT_FOUND
    next_action: ask_user

The card forces a useful review conversation. What makes this tool different from search_orders? Which values are trusted? What does “success” mean? What can safely happen next? If the answers are vague, the model is being asked to infer architecture from labels.

How can you test tool selection without fooling yourself?

Test the choice at the boundary, not only the final answer. The following seven-case matrix is a reusable test design. It does not claim a pass rate or a benchmark result.

Case	Prompt condition	Expected behavior
Ambiguous intent	“Find the order and tell me if I can cancel it”	Use a lookup path, then stop or request the missing cancellation facts; do not write immediately
Exact intent	“Cancel order ord_123 because it arrived damaged”	Choose the cancellation path only if policy and required fields allow it
Conflicting synonym	Use “look up,” “find,” and “retrieve” for the same known ID	Stable routing when the identifier and operation are unchanged
Missing identifier	Ask for one exact record without an ID	Search or ask for a disambiguating field; never invent an ID
Read/write boundary	Ask for the current address, then ask to change it	Read and write calls stay distinct, with the write guarded
Unavailable tool	Remove the intended tool from the candidate set	Explain the limitation or use a declared fallback; never pretend the tool ran
Wrong entity risk	Two customers have similar names	Require a trusted identifier or clarification before a consequential action

For each case, assert the tool name, normalized arguments, candidate set, authorization decision, side effects, and final response. Run the matrix across the model endpoint and configuration you actually deploy. One successful demo cannot tell you whether the boundary is reliable.

Keep this test focused on selection and routing. If the selected action then repeats without progress, use the AI agent loop guide. If you are turning these cases into a pre-release gate, use the AI agent evaluation guide for the broader release decision.

What evidence should you capture from a failing turn?

Start with the model-facing request, not the source repository. The function definition in your code may differ from the definition that reached the model after permissions, feature flags, namespaces, tool search, and request-level filters were applied. A routing diagnosis that omits that last representation is a guess.

Capture one complete failing turn in a redacted record. Keep the record small enough to inspect in a code review, but complete enough to replay the decision. The minimum fields are:

Field	Why it matters	Example
run and turn ID	Connects selection to execution and follow-up turns	run_41, turn_03
model and endpoint	Behavior depends on the deployed model and API path	model-name, dated endpoint
user request	Preserves the language that created the ambiguity	“Find the account and close it”
available tools	Shows what the model could see	search_accounts, get_account
allowed tools	Shows the final runtime filter	get_account only
definitions and schemas	Reveals rival wording and valid states	versioned JSON
selected call	Separates tool identity from arguments	search_accounts({query: ...})
authorization decision	Shows whether the call was permitted	read: allowed, write: denied
result and effect	Distinguishes response text from system state	pending, no state change

Log the candidate set after every filtering step if the system has more than one tool-loading stage. A single available_tools field may still hide the important transition. For example, a connector may advertise fifteen tools, a tenant policy may reduce that list to nine, a task router may select four, and a request-level allowlist may leave two. The model only sees the last list, but the earlier lists explain why a required tool disappeared or why an unrelated rival survived.

Record versions for definitions and policies. A tool name can remain unchanged while its description, required fields, permission rules, or result status changes. If the trace stores only get_order, a later investigator cannot tell whether the model saw the old contract or the new one. Treat the definition as a versioned interface, just as you would treat an HTTP response schema.

Redact secrets and personal data before the trace enters a shared system. Do not remove the evidence that identifies the routing boundary, though. Replace an email address with user_a@example.test, preserve whether an exact ID was present, and keep the relationship between the user, tenant, and target. A redacted trace can still answer the important question: did the agent have enough trusted information to choose this operation?

A wrong-tool diagnosis is only as good as the model-facing tool list captured in the failing trace.

Permissions should narrow the model's choices before selection and block the executor again before execution.

How can permissions and tenant context create a wrong choice?

Permissions do more than block execution. They often change the candidate set before the model chooses. A user who may read invoices but not issue refunds should not receive a model-facing definition that presents both operations as peers and relies on an error after selection. The executor must still enforce the rule, but the model should also see the truthful set for the current actor and task.

Tenant context creates a similar problem. One workspace may use archive_project, while another can only use export_project. If the system merges all connector definitions and applies tenant checks late, the model may choose a tool that exists in the general library but is not valid for this tenant. The resulting error can look like poor reasoning when the real issue is a stale or overly broad tool catalog.

Build the candidate set from trusted context in code. Resolve the actor, tenant, workflow stage, resource scope, and allowed side-effect level before the model call. Then expose only the operations that can be considered in that state. Keep a final authorization check at execution time because context can change between selection and execution.

There is a useful distinction between hiding and refusing. Hiding a tool prevents it from competing in the current choice. Refusing a call protects the system when the model or an attacker sends a call outside policy. You need both. Hiding reduces confusion; refusal is the security boundary.

Do not encode permission only in a natural-language sentence such as “never refund without approval.” That instruction may help the model, but it is not a reliable access control. The executor should inspect the authenticated actor, the target account, the requested amount, the approval record, and the current state. If any check fails, return a typed error and make no external change.

When access changes during a conversation, refresh the tool set. A cached list can make an old permission look current. Store the permission snapshot or policy version with the trace so an incident review can tell whether the model acted on current context. MCP’s tool-list notifications and tool discovery model make this kind of change explicit, but any provider can implement the same operational rule with its own metadata (Model Context Protocol Tools specification).

How do names, descriptions, and examples compete with one another?

A model does not experience a tool library as a neat API reference. It sees a set of tokens that must be matched to a request. Two definitions can be technically accurate and still be indistinguishable at the choice point.

Compare these descriptions:

find_customer: Find customer information.
get_customer: Retrieve customer information.

Both descriptions describe a useful capability. Neither tells the model what evidence separates one from the other. A better pair states the input condition and the boundary:

search_customers: Find candidate customer records when the request has no trusted customer ID.
Do not use this tool when an exact customer ID is already available.

get_customer: Return one customer record for a trusted customer ID.
Do not use this tool to search by name, email, or free text.

The names now reinforce the distinction. The descriptions explain the positive condition and the negative condition. The schema should repeat the same distinction in a machine-checkable way, with customer_id required for the second operation and a search query required for the first.

Examples can help, but examples do not rescue a contradictory contract. If the description says “use only with an exact ID” while an optional name field remains in the schema, the model has to resolve the conflict. Remove fields that do not belong. If a field is known by the application, pass it in code instead of inviting the model to regenerate it.

Review rival definitions together. A tool owner working alone may approve a perfectly clear description because the local function is clear. The routing problem appears only when that definition is placed beside five other “find,” “lookup,” “retrieve,” and “update” tools. Make a review table for each high-risk pair and require a one-sentence answer to “when should this tool lose?” If the team cannot write that sentence, consolidate the operations or insert a deterministic router.

The same rule applies to agent-as-tool designs. Names such as research_agent, knowledge_agent, and search_agent can overlap even when the underlying systems differ. Name the job and the boundary, not just the implementation. search_internal_policy and draft_customer_reply_from_policy are easier to separate than three generic agent names.

A tool description needs a clear losing condition, or its nearest rival will remain plausible.

When should routing move from the model into application code?

Move a decision into code when the available facts already determine the safe operation. A model is useful for interpreting a natural-language request, but it does not need to choose among several tools when the application already knows the workflow stage, resource type, or permission result.

Suppose a user enters an exact order ID into an order page and clicks “view status.” There is no benefit in asking a model to choose between search_orders, get_order, and cancel_order. The page can call get_order directly. A model might still summarize the returned record, but it should not decide which read operation is permitted when the interface has already made that decision.

Keep the model in the ambiguous part. It can classify “I want to know where my order is” as a status request, extract a date range from “orders from last week,” or ask a clarifying question when two targets are plausible. The application can then map that intent to a small allowed set and verify the arguments.

A practical split looks like this:

1. The application resolves identity, tenant, permissions, workflow stage, and known identifiers.
2. The model extracts intent and uncertain user-provided values.
3. The application validates those values and filters tools.
4. The model chooses among the small, remaining set when a real ambiguity remains.
5. The executor checks authorization, target state, and idempotency before the side effect.

This split reduces the cost of a wrong selection. It also makes the system easier to test. You can test the deterministic router with ordinary unit tests and reserve model evaluations for the parts that genuinely require language interpretation.

Do not move every decision into code just to avoid model behavior. An enormous collection of hand-written routes can be harder to maintain than a small model-facing surface. The test is whether the rule is stable, trusted, and already known. Stable rules belong in code. Ambiguous language belongs with the model, inside a constrained boundary.

How should read and write tools be separated?

Read and write operations need visibly different names, schemas, permissions, and execution paths. A generic manage_customer tool that can view, edit, merge, and delete records asks the model to infer several boundaries at once. It also makes review and authorization harder because the operation is hidden inside an argument.

Prefer separate operations such as get_customer, update_customer_email, and merge_customer_records. Make the write schema narrow. Require a stable target ID, include only fields that the operation can change, and add a reason or confirmation token when policy requires one. The executor should ignore unexpected fields rather than silently accepting a broad object.

A write tool should return enough information to establish what happened. “Done” is not a useful contract if a request was queued, partially applied, rejected by a downstream system, or accepted without changing state. Return a status, target ID, state version, changed fields, and a repeat rule. If the operation is pending, return the durable job ID and the permitted status-check action.

Use dry-run or preview operations for changes that are difficult to reverse. A model can propose a set of edits, the application can render the normalized plan, and a user or policy engine can approve it. The final executor must recompute or validate the plan rather than trusting a free-text approval that could refer to a different target.

The distinction is not cosmetic. MCP explicitly tells clients to treat tool annotations as untrusted unless they come from trusted servers. The specification also requires servers to validate inputs and implement access controls. The executor therefore needs its own decision boundary, regardless of what a tool description says:

“For trust & safety and security, clients MUST consider tool annotations to be untrusted unless they come from trusted servers.” Model Context Protocol Tools specification

A schema can make an invalid argument impossible, but only a policy boundary can make an unauthorized effect impossible.

What does a useful result contract look like?

The result is part of the routing interface. It determines what the model believes happened and which tool it considers next. A long paragraph written for a human operator may contain the needed facts, but it forces the model to extract status, identity, and next action from prose.

Use stable fields with a small vocabulary. The following is a provider-neutral shape, not a required vendor format:

{
  "status": "success",
  "code": "CUSTOMER_FOUND",
  "entity_id": "cus_123",
  "state_version": "v18",
  "changed": false,
  "next_allowed_action": "update_customer_email",
  "safe_to_repeat": true
}

For object inputs, make required properties and additional fields deliberate rather than accidental. The JSON Schema object guidance is useful here because it separates the shape of an object from the application rule that decides which values are trusted (JSON Schema object reference). The schema can reject an unknown field, but the application still has to decide whether the target is the correct customer or order.

Define what each status means. success should mean the requested operation completed, not merely that the server accepted the request. pending should identify a durable operation that can be checked. retryable_error should explain what can change before a retry. permanent_error should tell the model to stop, ask the user, or choose a declared fallback.

Return errors in a way the orchestrator can distinguish from normal content. The MCP specification describes tool execution errors and says clients should provide actionable errors to language models so they can self-correct. That does not mean every error should trigger a retry. Include a retry policy or next action so the model does not keep calling a tool with a new spelling of the same invalid argument (MCP Tools specification).

Keep the result scoped to the next decision. Do not return an entire database record when the next operation needs only the stable ID, current status, and one permission flag. Large results increase context pressure and may introduce irrelevant names that compete with the original target. Provide a detail tool when the user or workflow actually needs the full record.

Test results with stale and partial downstream states. A payment provider can report that a request is accepted while the account remains unchanged. A CRM can return a record that was deleted between lookup and update. A queue can retry a task after the model has already received a timeout. The result contract should expose enough state for the executor and orchestrator to choose a safe next step.

How should retries handle a wrong tool?

Retries can hide a routing defect. If the first call selects search_orders when the user supplied an exact order ID, blindly retrying with a slightly different query may produce a plausible record and make the final answer look correct. The system has still spent an extra turn and may have selected the wrong entity.

Classify the failure before retrying. A transport timeout, rate limit, or temporary downstream outage may be retryable without changing the tool. An invalid argument may need correction or a user question. A wrong tool is a routing failure and should usually return to the selection boundary, not loop inside the same tool.

Store the original selection and the reason for the retry. If the orchestrator changes from search_orders to get_order, that should be visible in the trace. If it retries the same operation, record why the repeat is safe and which state or input changed. Idempotency keys protect external effects, but they do not make a semantically wrong read useful.

Use retry budgets by failure class. For example, allow one correction for a malformed date, one status poll for a pending job, and no automatic retry for a denied write. The exact numbers depend on the workflow. What matters is that a new attempt requires new evidence rather than only another model turn.

If a tool returns an ambiguous result, prefer clarification over creative recovery. “I found three accounts with that name” is not a failure to be solved by guessing. Ask the user for a trusted identifier or show a safe selection list. A model that chooses one because it wants to complete the task is exhibiting the exact behavior the routing contract should prevent.

A retry should require new evidence, not merely a new wording of the same failed tool call.

How can you build a routing regression dataset?

A routing dataset does not need thousands of examples to become useful. Start with the smallest set that distinguishes the candidate tools. Each case should contain the user request, trusted context, expected candidate set, expected tool or clarification, argument constraints, authorization result, and expected external effect.

Build cases in pairs and triplets. A single “look up order” example cannot tell you whether the system understands the boundary. Add a near neighbor with no order ID, one with a similar customer name, and one that asks to cancel the order. The expected behavior should change only when the relevant fact changes.

Useful case families include:

• exact identifier versus a free-text description;
• read request versus write request;
• one matching entity versus several similar entities;
• permitted tenant versus restricted tenant;
• tool available versus tool absent;
• current state versus stale state;
• successful result versus pending result;
• retryable failure versus permanent failure;
• direct user request versus instruction embedded in retrieved content;
• one tool namespace versus two tools with overlapping names.

For each case, assert more than the final answer. A system can say the right thing after calling the wrong read tool, and it can say a safe thing after attempting an unauthorized write. Assert the candidate set, selected tool, normalized arguments, policy decision, downstream calls, side effects, and final response.

Keep an expected rationale for the dataset maintainer, but do not require the model to reveal private chain-of-thought. The test needs observable events and a concise classification such as correct_tool, clarification_required, tool_unavailable, argument_error, or policy_denied. This makes the dataset stable across providers and model versions.

Run the cases against the exact configuration you deploy. A routing result can change when the model, tool order, description, temperature setting, endpoint, or request wrapper changes. Store the configuration alongside the result. Do not compare a production trace with a local experiment and call the difference a model regression without checking the surrounding request.

Keep failures as permanent regression cases until the contract changes intentionally. When a case is removed, record why. Otherwise a team can “fix” a misroute by deleting the example that exposed it.

Which routing metrics should you monitor?

The first useful metric is selection accuracy on labeled cases, but production monitoring needs more context. Track the rate of correct tool selection, clarification, argument validation failure, policy denial, retry, and external side effect. Break those rates down by workflow, tenant, model endpoint, tool version, and risk level.

Measure the cost of a misroute. A wrong read may add latency and a second call. A wrong write may create a customer-visible change. A tool that is selected correctly but returns a misleading status may cause more harm than an obvious selection error because the trace looks successful.

Useful operational signals include:

Signal	What it reveals
candidate-set size	Whether the model is comparing too many tools
rival selection rate	Which tools steal requests from one another
clarification rate	Whether boundaries are clear or overly narrow
schema rejection rate	Whether arguments express valid states
policy denial rate	Whether exposure and authorization disagree
retry by failure class	Whether the orchestrator repeats the wrong layer
pending-to-success time	Whether result semantics match workflow reality
side effects after denial	Whether the executor fails closed

Do not optimize only for fewer clarifying questions. A lower clarification rate can mean the model is guessing more often. Set an acceptable tradeoff by workflow. For a low-risk internal lookup, one extra question may be annoying. For a refund, permission change, or deletion, a higher clarification rate may be the correct result.

Review samples, not only aggregate numbers. A stable overall rate can hide one tenant whose tool filter is broken or one high-risk operation that fails rarely but matters greatly. Attach a compact trace to each alert and redact it before wider access.

Treat the monitoring record as part of the system's risk evidence. The NIST AI Risk Management Framework is a useful reference for keeping risk identification, measurement, and response connected instead of treating a successful final answer as the whole outcome.

How should you roll out a routing repair?

Change one layer at a time when the failure is still understandable. Freeze a baseline using the original trace and the nearby regression cases. Then change either the candidate set, rival descriptions, schema, result contract, or executor policy. Run the same cases and compare selection, arguments, effect, latency, and clarification.

Start in shadow mode for consequential workflows. The repaired router can produce a proposed tool and normalized arguments while the existing path remains responsible for execution. Compare the proposals without sending the new side effect. This reveals whether the repair changes more requests than intended.

Use a small allowlist during the first live release. Expose the repaired definition only to one workflow or tenant, record the candidate set, and keep an immediate fallback that does not repeat a risky call. Monitor policy denials and unexpected clarifications as carefully as successful calls.

Expand only after the regression cases stay green across the deployed model and configuration. Then update the definition version, test fixtures, and operational notes together. A description change is a behavior change, even if the function signature is identical.

Have a stop rule. Pause the rollout if a write is selected for a read-only request, if a target ID changes between lookup and execution, if an unavailable tool is presented as completed, or if a retry creates a duplicate external effect. The stop rule should be executable by the system where possible, not dependent on someone noticing a confusing transcript.

What belongs in a wrong-tool incident review?

An incident review should answer five questions in order:

1. What did the user ask, and what trusted context was present?
2. What tool definitions and candidate set reached the model?
3. Which tool and arguments did the model select?
4. What did the executor authorize, run, and return?
5. What repair will prevent the same boundary failure without hiding a useful capability?

Avoid beginning with a verdict about the model. That framing encourages a prompt edit before the team has established the failure layer. Use neutral labels such as unavailable capability, rival ambiguity, schema mismatch, result ambiguity, policy gap, or downstream state mismatch.

The review should include the smallest reproduction, the original and repaired definitions, the relevant policy version, and the regression cases. Include the final external state, not just the assistant response. If the operation was reversible, record the reversal and whether it actually restored the prior state.

Separate immediate containment from durable repair. Immediate containment may disable a write tool, require approval, or route the workflow to a human. Durable repair may split a broad tool, change the result status, or move stable routing into code. Do not call containment a fix if the same ambiguous surface remains waiting behind a temporary flag.

Record uncertainty. If logs do not show whether a tool was available, say so. A missing trace field is itself a reliability issue. Add the field and keep the incident open until a future run can distinguish the competing explanations.

How do provider differences change the repair?

The concepts in this guide are provider-neutral, but the controls are not identical. One API may expose forced tool choice and an allowed subset. Another may emphasize detailed descriptions and tool consolidation. An MCP client may discover tools dynamically and receive structured results. An SDK may derive descriptions from docstrings and schemas from function signatures.

Use provider documentation to verify the control you plan to use. Do not assume that a parameter named tool_choice, allowed_tools, strict, or deferred has the same scope across SDKs. Record the actual serialized request in a trace and test the behavior at the model endpoint you deploy (OpenAI Function calling guide, OpenAI Agents SDK Tools).

Keep your internal contract above the provider layer. Define concepts such as candidate_tools, selected_tool, normalized_arguments, authorization, result_status, and next_allowed_action in your own trace schema. Map provider-specific events into those fields. This lets you compare a routing failure across systems without pretending that every provider offers the same native feature.

Provider guidance can also change. Tool-search behavior, schema support, model limits, and SDK defaults are moving parts. Set a review date for high-risk claims, pin the documentation version when the provider offers one, and make a change to the tool surface trigger the routing test suite.

The stable principle is simple: a model chooses from the representation it receives. Make that representation narrow enough to inspect, distinct enough to choose, and constrained enough that a bad choice cannot create an unauthorized effect.

What usually fails as a fix?

“Just add a stronger system prompt”

A prompt can clarify a real ambiguity. It cannot make an unavailable tool available, repair a misleading schema, or enforce an authorization rule. Change the model-facing contract and the runtime boundary when those are the failing layers.

“Turn on strict mode”

Strict schemas are useful for argument shape. They do not turn tool selection into a deterministic classifier. Keep the selection test separate from the argument-validation test.

“Give the model every tool so it can choose”

More capability can create a harder routing problem. OpenAI recommends evaluating the number of initially available functions and using tool search for large surfaces; Anthropic recommends fewer, more capable tools and meaningful namespacing (OpenAI’s function-calling guide, Anthropic’s tool-definition guidance). If a tool is rare or irrelevant in the current stage, do not make the model compare it on every turn.

“Add a second model to judge the first”

A second model can be useful in a deliberately tested architecture, but it also creates another routing decision, another latency and cost path, and another failure mode. First make the candidate set, contracts, and policy checks legible. A deterministic filter is often easier to inspect than an extra opinion.

“Log only the final answer”

The final answer may sound correct after the wrong tool returned plausible data. Save the candidate set, selected tool, arguments after validation, result status, and verified effect. Without those fields, you are debugging a story instead of a decision.

Should you fix it yourself or ask for help?

Handle it internally when the tool list is small, the failure is reproducible, the side effects are reversible, and one team owns the schemas and executor. Start with the WRONG framework, make one contract change, and run the seven cases before changing several variables at once.

Ask for an architecture or reliability review when the tool surface spans several teams, the wrong choice can send messages or change records, the available set differs by user or tenant, or nobody can reconstruct what the model was allowed to call. The useful review packet is small: one failing trace, the model-facing tool definitions, the permission decision, the tool result, and the expected effect.

Marius’s one-to-one AI consulting page describes the kind of work that can be done on a real workflow. The article is complete without that step. If you ask for help, bring the trace and the contract card rather than a screenshot of the final answer.

The shortest reliable rule is this: make the right tool the clearest available choice, make the wrong tools unavailable or visibly different, and make the executor verify the choice before it can matter.

For the next step, take one real wrong-tool trace and fill in the five WRONG gates. If you cannot answer “what was available?” or “what could safely happen next?”, you have found the repair boundary.