Isolation starts with identity, not a system prompt
A prompt saying that the assistant must only use the current company’s data is not a security boundary. Access control must run before any content reaches the model. The entry service should authenticate the user, derive the tenant from trusted credentials, and create tenant context that the client cannot override. A tenant_id supplied in a request may be checked for consistency, but it should never grant access by itself.
That context must follow the request through relational queries, vector retrieval, object storage, caches, queues, tool calls, and audit records. A single omission can expose another tenant’s information. Typical failures include applying metadata during indexing but not retrieval, caching conversation summaries without a tenant key, or running background jobs under an unrestricted service account. Make tenant scope a required argument at repository and tool boundaries. Missing scope should cause a closed failure, never a global query.
Choose shared or dedicated resources by risk and operating cost
There are three useful patterns: shared infrastructure with logical isolation, dedicated infrastructure per tenant, and a hybrid. Shared databases and vector indexes simplify operations and make efficient use of capacity, but every query, maintenance task, and deletion workflow must enforce tenant filters. Dedicated databases, indexes, or cloud accounts create clearer permission and failure boundaries, while increasing deployment, upgrade, monitoring, and capacity-management work.
Tenant count alone should not decide the model. Consider data sensitivity, residency obligations, customer-managed encryption keys, the authorization model of connected systems, and whether one tenant’s workload could degrade others. A practical platform may use shared resources for ordinary knowledge bases while assigning dedicated storage or compute to regulated workloads or stricter contracts.
- Data: Tables, vectors, files, backups, derived summaries, and indexes need verifiable ownership and a complete tenant-deletion path.
- Secrets: Separate integration credentials and, when required, encryption keys so one connector cannot reach another organization’s ERP, CRM, or cloud resources.
- Compute: Apply tenant quotas, concurrency limits, and timeouts so bulk ingestion or long-running agents cannot consume shared capacity unchecked.
- Tools: Build the available tool set from tenant and user permissions. The model should never select an unauthorized tool or supply tenant identity on its own.
- Observability: Tag traces and cost records by tenant, while redacting prompts, personal data, access tokens, and sensitive retrieved passages.
Context management is selective assembly, not unlimited memory
Useful context is the smallest set of information needed for the current task. Separate request-scoped execution state, short-term conversation memory, user-approved preferences, and tenant knowledge from documents or business systems. Each category deserves its own retention period, write rules, and deletion process. Conversation summaries should not automatically become facts: they can preserve earlier assumptions or model errors and quietly reintroduce them later.
Use a deterministic assembly pipeline for every turn. Verify tenant and role first, load only necessary conversation state, retrieve authorized evidence for the current question, and then add tool results. Assign explicit token budgets to each source, preserving the current request, authorization constraints, and traceable evidence before older dialogue. Retrieved documents should retain source and permission metadata and be treated as untrusted input; instructions embedded in a document must not override system policy. Before writing long-term memory, decide whether it is necessary, sensitive, reviewable, and deletable by the user.
Test across boundaries, including asynchronous paths
Normal functional tests show that the assistant works; they do not prove isolation. Seed each test tenant with recognizable private content, then attempt access from another tenant through exact keywords, semantic similarity, continued conversations, identical filenames, cache reuse, manipulated tool parameters, and prompt injection. Inspect citations, model inputs, tool calls, streaming events, and traces as well as final answers. Data can leak into a model or log even when it is absent from the displayed response.
Document parsing, embedding, summarization, and agent jobs often run asynchronously, where tenant context is easiest to lose. Queue messages should carry signed or re-verifiable identity, and workers should reacquire least-privilege access instead of trusting payload fields. Before launch, exercise tenant suspension, deletion, key rotation, index rebuilding, and backup restoration to ensure derived data is neither retained nor restored into the wrong boundary. When the assistant spans LINE, ERP, CRM, and cloud services, an integration team should review the complete data path rather than treating the chat interface as the system boundary.