Define what stale actually means
In an enterprise knowledge base, stale content is not simply old content. The more dangerous case is content that still looks authoritative but no longer matches the current process, system behavior, contract language, or operational rule. An AI assistant may retrieve that document, answer confidently, and create a decision that is technically grounded in a source but wrong in the real business context.
We normally separate knowledge into a few classes: stable principles, policies that change on a schedule, procedures tied to system versions, and live operational data. These should not share the same maintenance rhythm. A service overview or architecture principle may only need periodic review. ERP field mappings, CRM workflow rules, LINE integration steps, cloud deployment procedures, and pricing conditions need to change when the actual workflow changes.
The practical question is not whether a document is old. The question is whether it can still support a correct decision. A two-year-old design principle may still be valid. A document updated last week can still be stale if it was revised by someone who did not check the actual system.
Put ownership where the truth lives
Many knowledge bases fail because ownership is assigned to the wrong team. The AI or data team can manage ingestion, chunking, retrieval, access control, evaluation, and monitoring. They cannot sustainably decide whether every sales reply, support procedure, security policy, accounting rule, or ERP operation is still correct. Content ownership has to sit with the team closest to the source of truth.
Each knowledge area should have a named owner, update triggers, and an approval path. A broad department label is usually too vague. When something breaks, everyone assumes someone else will fix it. A better pattern is to connect knowledge maintenance to existing change processes: product releases, SOP changes, API changes, contract updates, customer support revisions, or internal tooling changes.
- System documentation:Owned by engineering or operations, updated when releases, APIs, schemas, or deployment steps change.
- Sales and support knowledge:Owned by revenue operations or support leads, updated when positioning, workflows, qualification rules, or response standards change.
- Compliance and security content:Reviewed by legal, security, or management owners so the AI does not repeat outdated policy.
- Integration rules:Owned by people who understand the ERP, CRM, LINE, data platform, and handoff process, because these errors are often invisible from the document alone.
The engineering responsibility is to make updates low-friction. If changing a FAQ requires a developer deployment, the content will lag. If owners can update through a document repository, CMS, Git workflow, or internal console with review controls, maintenance has a chance to become routine.
Use change triggers, not only scheduled reviews
Scheduled reviews are useful, but they are not enough. The events that make a knowledge base unreliable are usually process changes and system changes. A stronger approach is to place update triggers where those changes already happen. Add documentation checks to release checklists. Recheck knowledge base entries when CRM fields change. Rerun test questions when a LINE bot flow is redesigned. Review integration notes when an ERP export format is modified.
For a RAG system, updating the source document is only part of the work. The pipeline also has to extract the content, split it, embed it, update the index, clear stale cache, and preserve permissions. In many AI assistant projects, the model is blamed for failures that actually come from a weak data pipeline. A knowledge base that powers production workflows should be operated like a production system.
We prefer two maintenance rhythms: event-driven updates and regular health checks. Event-driven updates handle high-risk changes such as process rules, pricing conditions, system fields, contract language, permissions, or customer-facing procedures. Regular health checks clean up slower problems such as duplicated pages, inconsistent wording, obsolete versions, ownerless documents, and content that answers the same question in conflicting ways.
Validate with real questions
A knowledge base is not valuable because it contains many documents. It is valuable when users can get correct, usable answers. That means maintenance should include a set of representative questions used to test whether the assistant retrieves the right source, answers within policy, cites the correct material, admits uncertainty, and escalates when the knowledge base does not contain enough information.
The test set should include normal questions and edge cases. Users will ask with outdated terminology, mix two system names, describe a vague goal, request information they are not allowed to see, or ask about something that is not documented. These cases reveal gaps in retrieval strategy, content quality, access control, and prompt behavior.
User feedback is also useful, but it should be categorized carefully. A negative rating does not always mean the answer was factually wrong. The source may be unclear, the user may have asked an ambiguous question, the retrieval step may have selected the wrong page, or the assistant may have failed to ask for a missing condition. Separate feedback into content errors, retrieval errors, permission errors, and question-understanding errors. Otherwise the team may rewrite documents when the real fix is indexing, access control, prompt design, or system integration.
Treat the knowledge base as part of the system
If the knowledge base is only a bundle of files uploaded into a vector database, it will eventually drift away from real operations. A more durable design separates sources by behavior. Stable explanations can live in a document repository. Live status should come from ERP, CRM, ticketing systems, IoT data platforms, or other operational systems. User permissions should come from the enterprise identity model. AI should not answer every question from static text.
There is a real trade-off. A document-based knowledge base is faster to launch and easier to maintain at first, but it requires disciplined ownership and review. System integration reduces stale-answer risk, but it adds engineering work, permission logic, exception handling, and monitoring. Not every answer deserves an API call. But when the answer depends on customer status, inventory, device data, approval state, current pricing, or user permissions, static documents are the wrong source of truth.
Keeping a knowledge base fresh is not a one-time cleanup. It is an operating model: named owners, change triggers, indexing observability, evaluation questions, user feedback, and selective integration with live systems. With that foundation, an AI assistant can move beyond a demo and become a tool the organization can trust in daily work.