A Model Response Is Not an Authorized Command
Once an enterprise assistant connects to an ERP, CRM, messaging channel, ticketing platform, or IoT service, tool calling becomes an operational interface. A model may be able to create an order, update customer data, send a LINE message, retrieve restricted records, or issue a device command. Valid JSON only proves that an argument can be parsed. It does not prove that the requested action is correct, authorized, or appropriate for the system's current state.
Engineering teams should therefore treat every tool call as an untrusted external request. Prompts and tool descriptions are useful for reducing mistakes, but they are not security controls. A model can misunderstand a date, confuse two customer identifiers, reuse stale context, or follow an injected instruction found in retrieved content. The dependable boundary is a deterministic validation and execution layer between the model and production services.
Validate Shape First, Then Validate Meaning
The first layer is structural validation. Give each tool a narrow, explicit input schema covering required fields, types, lengths, formats, enumerations, and whether additional properties are accepted. A monetary amount should not be free-form text. Timestamps should carry an agreed format and time zone. Identifiers should match the conventions of the target system. Rejecting unknown properties is usually safer than silently discarding them because they may represent either model invention or an attempted bypass.
A schema, however, only answers whether data has the right shape. After parsing, apply semantic and business validation. Confirm that the customer exists, the product is active, the requested delivery date is allowed, a refund does not exceed its source transaction, and a device command is supported by that device model. Cross-field rules matter too: legal entity, currency, payment method, tax treatment, and destination may be individually valid while forming an invalid combination.
- Types and formats: Reject ambiguous numbers, unparseable dates, oversized strings, and unexpected nested objects.
- Allowed values: Use allowlists for actions, roles, statuses, and transitions instead of accepting model-created labels.
- Resource existence: Re-query customers, orders, devices, and products from trusted systems rather than relying on conversational context.
- Business invariants: Enforce limits, workflow states, inventory rules, and relationships already protected by the core application.
- Normalization: Standardize time zones, phone numbers, addresses, and identifiers after validation so downstream services receive predictable data.
Bind Authorization to the Actor, Resource, and Action
Making a tool available does not authorize every user to run it. The validation layer must use identity and authorization context established by the application, never a userId, role, or tenantId supplied by the model. The backend should evaluate who is acting, which specific resource is targeted, and what action is requested. In multi-tenant applications, derive tenant scope from the authenticated session so generated arguments cannot select another organization's records.
Authorization should also reflect the consequence of the action. A read-only query may run immediately after access checks. Sending an external message, changing master data, or creating a low-risk draft may require a preview and explicit confirmation. Payments, refunds, deletions, bulk changes, and device control often justify approval workflows or separation of duties. A confirmation screen should display the actual normalized values that will be submitted, not merely ask the user whether they want to continue.
Make Execution Retry-Safe Without Duplicating Effects
Validation does not eliminate timeouts, retries, or partial failures. AI applications may repeat a tool call after losing a response, so any operation with side effects needs an idempotency key. Store a request fingerprint, execution status, and result at the backend boundary. If the same completed operation arrives again, return the recorded outcome instead of creating another order, sending another notification, or applying the same adjustment twice.
For workflows spanning several systems, do not assume distributed changes will remain consistent automatically. A safer pattern is to create a pending operation, execute steps through a durable queue, expose explicit states, and define compensation for failures that cannot be rolled back directly. Audit records should capture the tool name, normalized arguments, authenticated actor, validation decision, approval evidence, and backend result. Secrets and sensitive fields still need redaction. These records support troubleshooting, compliance review, and incident investigation.
Fail Safely and Give the Model a Controlled Recovery Path
When validation fails, do not return database details, internal field names, credentials, or stack traces to the model. Use stable error codes with a bounded explanation of what can be corrected, such as a missing time zone, an order state that does not permit refunds, or insufficient access to the selected resource. Only retryable errors should invite the model to repair its arguments. Authorization failures, high-risk decisions, and repeated invalid attempts should stop automatic execution and return control to the user.
Testing should go beyond well-formed examples. Exercise missing and extra fields, extreme lengths, wrong-tenant identifiers, stale resources, duplicate requests, injected instructions, backend timeouts, and failures between workflow steps. Verify that rejected calls produce no side effects. In production, monitor rejection reasons, retry patterns, and human approval outcomes to improve schemas and prompts while keeping enforcement in deterministic code. When an assistant spans multiple enterprise platforms, a shared validation and execution gateway is usually easier to govern than separate safeguards added independently to every integration.