Classify the impact, not the error message
An AI service can return successful HTTP responses while producing outdated guidance, exposing restricted documents, or sending incorrect instructions to an ERP system. Severity should therefore consider availability, confidentiality, answer integrity, transaction safety, affected users, and the likelihood that harm will continue to spread.
Use a small number of severity levels with predefined owners, notification paths, and emergency permissions. Reassess the level as evidence changes. When the team cannot yet rule out sensitive-data exposure or irreversible actions, treating the event as the higher severity is usually the safer operational choice.
- SEV-1:Data exposure, unauthorized actions, loss of a critical workflow, or continuing high-risk automation. Disable the affected capability and activate incident command immediately.
- SEV-2:Major functionality is impaired, answer quality is broadly unreliable, or a critical integration is repeatedly failing, but a usable fallback exists.
- SEV-3:The impact is limited to certain users, document collections, or noncritical features, with a stable workaround available.
- SEV-4:A minor presentation defect, low-risk intermittent error, or warning that has not yet affected users. Track it through normal maintenance with explicit escalation conditions.
Contain damage before pursuing the root cause
The first operational goal is a single command structure and a trustworthy timeline. Assign an incident commander for decisions, a technical lead for investigation, a recorder for evidence, and a communications owner for consistent updates. Pause unrelated deployments and capture model versions, prompts, indexes, permissions, feature flags, and provider status before the environment changes further.
- Isolate:Revoke suspicious credentials, block an affected tenant, stop tool calls, or place the assistant in read-only mode.
- Degrade:Replace generated answers with document search, fixed guidance, human handoff, or an existing form-based workflow.
- Roll back:Do so only when the previous version and its data compatibility are understood. Treat the model, prompt, retrieval index, and parser as one release unit.
- Fail over:A backup model provider may restore capacity, but its data handling, safety behavior, tool calling, and output format must be verified first.
Choose containment by comparing the cost of degradation with the risk of continued harm. A support assistant that stops answering may create a queue, but that is often preferable to issuing incorrect refund guidance. Low-risk knowledge search might remain available while every operation that writes to CRM or ERP is disabled.
Use an AI-specific diagnostic path
Traditional monitoring exposes latency, errors, and resource pressure, but it rarely tells the team whether an answer is trustworthy. Break each request into identity and authorization, retrieval, prompt assembly, model inference, tool execution, and response processing. Connect those stages with one trace identifier while masking sensitive content and enforcing appropriate log retention.
Do not reproduce the incident with only one captured prompt. Build a de-identified replay set covering ordinary queries, permission boundaries, missing knowledge, prompt injection, provider timeouts, and partially completed tool calls. If changing the model removes the symptom, that does not prove the model was the cause; context length, retrieval ranking, schema parsing, or provider constraints may have changed at the same time.
- Incorrect answers:Inspect document versions, chunking, retrieved passages, citation mapping, and the no-answer policy.
- Authorization failures:Verify that access filters run before retrieval and that caches and vector indexes preserve tenant boundaries.
- Wrong actions:Check tool arguments, idempotency keys, approval gates, and compensation for partial completion.
- Latency or cost spikes:Look for retry loops, expanding context, fallback chains, batch jobs, and external rate limits.
Recovery requires evidence, not a successful restart
After applying a fix, replay the incident and core regression cases in an isolated environment, then restore traffic within a controlled scope. Validate citation accuracy, authorization, tool side effects, timeout behavior, cost limits, and audit records as well as basic response success. If a high-risk capability cannot be proven safe, restore read access first and keep writes or autonomous execution disabled.
Before closure, confirm that operational signals are stable, queued work is reconciled, incorrect transactions are compensated, affected data is understood, and users receive a consistent explanation. The review should document the timeline, impact, trigger, contributing conditions, effective and ineffective responses, and an owner plus verification method for every corrective action. Its purpose is system improvement, not individual blame.
- Immediate:Add missing alerts, remove dangerous defaults, cap retries, and provide an operational degradation switch.
- Near term:Add authorization-boundary tests, version prompts and indexes, maintain replay suites, and rehearse provider failure.
- Structural:Reduce blast radius and place high-risk actions behind approval, idempotency, compensation, and complete audit trails.