Define the data boundary before writing detectors
The first engineering question is not how to recognize an email address. It is which data may be sent to which model, for what purpose, and under whose authority. Names, phone numbers, national identifiers, and payment details are obvious candidates. Customer codes, quotations, device serial numbers, source code, contract terms, and unpublished operating information may be equally sensitive even though they are not conventional personal data.
Start with a classification and purpose matrix. For each data class, record its source, permitted model environments, acceptable transformations, blocking conditions, and whether restoration is required after generation. A self-hosted model, a controlled enterprise endpoint, and a general external API need not share the same policy. Consider data minimization, retention settings, regional requirements, user permissions, and the semantic detail the task genuinely needs.
Use layered detection
No single detector is reliable enough. Regular expressions work for stable formats, but can mistake an order number for an account number or miss values containing spaces, full-width characters, and line breaks. Named-entity models find people and addresses in prose but rarely understand company-specific identifiers. A practical pipeline normalizes input first, then combines patterns, checksum validation, field structure, dictionaries, entity models, and contextual rules.
- Format layer: Detect emails, phone numbers, IP addresses, credentials, financial accounts, and common identifiers, validating checksums where possible.
- Structure layer: Use JSON keys, database columns, form labels, document sections, and MIME types as evidence.
- Semantic layer: Identify names, addresses, HR or medical statements, and confidential information without a fixed format.
- Business layer: Maintain product codenames, internal project names, customer codes, and restricted terms.
Each finding should carry its type, source span, detector, confidence, and policy outcome without placing the raw value in ordinary logs. Merge overlapping spans before replacement and apply the stricter classification when detectors disagree. For long documents, preserve position mappings across parsing and chunking; otherwise, identifiers split across table cells or page boundaries can escape detection.
Choose a transformation that preserves the task
Removing every sensitive value is safe but can destroy the relationships the model needs. A support summary may need to know that the same customer appears in several messages, while not needing the customer's real name. Stable typed tokens such as PERSON_1, PHONE_1, and ORDER_1 preserve those relationships. Scope tokens to the smallest useful boundary so that identifiers cannot be correlated across unrelated users, tenants, or sessions.
Useful policy actions include allow, generalize, irreversible redact, reversible tokenize, and block. A reversible mapping should live only in a controlled service, be encrypted, expire quickly, and be bound to the tenant, user, and request. The model must not receive a restoration tool. If a task needs only geographic context, reduce a street address to a city or region. Passwords, private keys, and access tokens should normally be blocked and routed into a security workflow rather than masked and passed onward.
Protect every path into a model
Place the masking service in a centralized gateway between applications and model providers, not only in a chat interface. The gateway should inspect user text, parsed attachments, conversation history, retrieved RAG passages, and tool arguments before constructing a model request. This prevents background jobs, batch summaries, or newly added interfaces from bypassing controls and gives teams one enforcement point across AWS, GCP, and multiple model vendors.
RAG requires controls at two moments: before content is embedded and before retrieved passages are sent to generation. If a source document may not reach an external embedding service, mask it before indexing or use an approved embedding environment. Inspect retrieval results again because document combinations, stale indexes, and changed permissions create new exposure paths. Apply the same rule to tool calls: email bodies, CRM filters, ERP update fields, and generated API arguments should be checked before execution.
Operate for error cost and observability
Treat detection as a maintained system rather than a one-time feature. Regression data should cover realistic synthetic values, spacing variations, full-width characters, misspellings, multiline text, tables, OCR noise, and negative examples such as product codes that resemble financial identifiers. Measure missed detections and excessive masking by data class, along with blocking reasons and whether the model can still complete its task. Error costs differ: missing a private key is generally more serious than misclassifying an ordinary sequence number.
Operational logs should retain the policy version, finding classes, actions, model destination, and processing latency, but not the original values. Keep any samples required for investigation in a separate environment with tighter access and explicit retention. Run new rules in shadow mode before enforcing them, then review which requests would be blocked. Finally, inspect model output as well: retrieved documents and tool results can reintroduce sensitive data downstream. A mature pipeline does not promise zero errors; it gives every exception a defined policy, owner, and verifiable handling path.