Define “inconsistent” before changing the prompt
When users report that an assistant is “sometimes right and sometimes wrong,” first identify what is actually changing. Does the same question produce different facts? Do different users receive different documents? Are the facts correct but the required format varies? These symptoms point to different failure modes, including sampling, access-control filters, retrieval, and output validation. Editing the prompt immediately may change the symptom without exposing its cause.
Create a minimal reproducible case. Capture the original input, user identity and permissions, conversation history, timestamp, model and version, sampling settings, system-prompt version, retrieval query, retrieved documents, tool responses, and the complete context sent to the model. Hold these variables constant and rerun the case, then change one variable at a time. Lowering temperature can reduce variation in wording, but it cannot repair stale documents, missing context, or a failing ERP endpoint.
Isolate each layer of the request pipeline
An enterprise assistant is usually a pipeline rather than a single model. It may include authentication, query rewriting, vector search, permission filtering, prompt assembly, generation, tool calls, and output validation. Debug the intermediate artifacts at each boundary. The first useful question is, “What information did the model actually receive?” The second is, “What did it do with that information?”
- Input layer: Check whether the web client, LINE channel, CRM, or ERP connector truncates text, changes encoding, removes formatting, or mixes conversation state between sessions.
- Retrieval layer: Inspect the generated search query, candidate documents, ranking, chunk boundaries, metadata, document versions, and permission filters. A model change will not help if the authoritative document never enters the candidate set.
- Context layer: Confirm that token limits are not removing critical rules. Look for duplicate passages that dilute the signal and conflicting policies placed in the same prompt without effective dates or precedence.
- Generation layer: Record model versions, temperature, top-p, output limits, and structured-output settings. A provider migration or changed default can produce behavior drift even when application code is unchanged.
- Tool layer: Log requests, responses, timeouts, retries, and cache hits. If an ERP lookup fails and the application silently asks the model to continue, an integration failure appears to users as hallucination.
Use controlled substitutes to narrow the fault domain. Send a known-good set of documents directly to the model, bypassing retrieval. If answers become stable, focus on search, ranking, or authorization. If they remain unstable, freeze the model response and test parsing, business rules, and presentation separately. This takes more discipline than changing retrieval, prompts, and models together, but it preserves the evidence needed to know which fix worked.
Replace impressions with a repeatable evaluation set
A team cannot manage quality by reading a few responses after each release. Build a durable evaluation set from real workflows. Include common questions, paraphrases, insufficient-information cases, cross-document questions, outdated material, authorization boundaries, and prompt-injection attempts. For each case, specify acceptable sources, required facts, prohibited content, and the expected behavior when evidence is missing. Sometimes the correct response is to request more information or state that the answer cannot be verified.
Score separate stages instead of assigning only one subjective quality label. Track whether retrieval found the right source, whether the response is supported by that source, whether citations contain the claimed facts, whether tool arguments are valid, and whether the result follows the required schema. Deterministic checks work well for JSON structure, identifiers, dates, and mandatory fields. Model-based evaluation can help triage semantic issues, but critical cases still require human review. Evaluator prompts and model versions must also be recorded because evaluators can drift.
Apply the fix at the failing layer
Match the remedy to the evidence. Poor retrieval recall may require better chunking, metadata, hybrid search, or query rewriting; poor ordering may justify a reranker. Conflicting documents require authority, version, and effective-date rules, not instructions telling the model to guess which policy is current. If prompts are too long, remove repetition and make precedence explicit. Workflows that trigger transactions should use structured outputs, field validation, and an explicit failure path rather than relying on the model to obey a formatting request.
- For factual consistency: Reduce sampling variance, require supporting sources, and prevent unsupported completion when evidence is absent.
- For natural language: Preserve limited generation flexibility, but separate fact selection from final phrasing.
- For live operational data: Treat tool results as the source of truth, define timeout behavior, and never let model memory silently replace a failed lookup.
- For high-risk actions: Add policy checks, authorization, human confirmation, idempotency, and an auditable record outside the prompt.
Production observability should connect each request to configuration versions, retrieved-document identifiers, tool outcomes, latency, errors, and user feedback while keeping sensitive data out of logs. Run the evaluation set before release, deploy changes gradually, and retain a rollback path. When the system spans LINE, ERP, CRM, cloud services, and internal knowledge stores, an integration team can help make the entire chain observable and testable; repeatedly replacing the model is rarely a substitute for that engineering work.