Separate email ingestion from attachment processing
A mailbox listener should not parse attachments and write directly to an ERP, CRM, or document repository. Its first responsibility is durable ingestion: capture the message identifier, sender, received time, subject, and attachment metadata; store the original message and files in controlled object storage; then publish a processing job. Validation, classification, extraction, and delivery can run as independent workers. If OCR, a model endpoint, or an ERP is unavailable, new messages can still be accepted without being trapped in a mailbox connection.
Give every attachment a stable processing key, typically derived from the message identifier, attachment index, and a content hash. A filename is not enough: senders reuse filenames, mail servers may redeliver messages, and users frequently resend corrected documents in the same thread. Downstream operations should be idempotent. Before creating a record, the integration checks whether that processing key already completed and decides whether to create, update, or skip. This is what makes retries safe rather than merely automatic.
Validation requires layers, not an extension check
An attachment should pass security and structural checks before business classification. The filename extension, declared MIME type, and actual file signature may disagree, so use reliable content detection. For archives, limit expanded size, file count, and nesting depth. Encrypted documents or macro-enabled files should follow an explicit quarantine policy instead of being forcefully parsed. Malware scanning, content parsing, and sensitive-data checks should produce separate results so operators can understand which control rejected a file.
Validation must also distinguish permanent failures from transient ones. An unsupported format, corrupt document, or password-protected PDF will not improve after another attempt. A scanner timeout or temporary storage failure might. A practical validation chain usually covers:
- Source controls: expected sender domains, approved mailboxes, forwarding paths, and available email-authentication results.
- File controls: detected type, size, content hash, malware result, and archive safety limits.
- Structural controls: required columns, worksheet names, field types, date formats, character encoding, and parseability.
- Business controls: known customer codes, order references, device identifiers, and acceptable document dates.
Classification should expose confidence and evidence
Rules are effective when conditions are stable and explainable: a dedicated sender address, a known filename pattern, a PDF field, or a spreadsheet header. Model-based classification is useful for quotations, invoices, or service reports whose layouts and wording vary, but its output should not be treated as verified fact. A balanced pipeline applies deterministic rules first, then uses a model for ambiguous content. The result should include the proposed document type, confidence band, supporting evidence, extracted fields, and the version of the rules or model used.
Confidence thresholds should reflect the cost of a wrong action. Sending an ordinary attachment to manual review usually causes delay. Misclassifying a purchase order as an invoice and posting it into the wrong workflow can create an accounting problem. High-confidence decisions may proceed automatically; an uncertain middle band should enter a review queue; low-confidence or unknown documents should be isolated. When a reviewer corrects a result, retain the original decision, final category, and reason for the change. That history supports audit requirements and provides useful material for improving rules or evaluating future models.
Design failure queues for recovery, not storage
Use separate paths for retryable failures, permanent failures, and human review. Transient errors should retry with exponential backoff, randomized delay, and a maximum attempt count so a struggling dependency is not overwhelmed. Each attempt should record the failed stage, normalized error type, a safe response summary, and a correlation identifier. After the limit is reached, move the job to a failure queue instead of allowing an infinite loop.
Before replaying a job, confirm that the underlying problem has been addressed and resume from the safest completed stage. If malware scanning, parsing, and extraction already succeeded but the ERP write failed, repeating download and OCR only adds cost and creates new failure opportunities. Replay records should also identify the code, rule, and model versions used, especially when a deployment changes classification behavior.
- Monitor backlog health: track queue depth, age of the oldest job, time spent at each stage, and changes in error categories.
- Provide operator controls: allow authorized staff to inspect the source file, correct fields, skip a job, replay it, and record a resolution note.
- Protect document data: restrict access to raw and extracted content, apply retention policies, and keep personal or confidential data out of routine logs.
Before release, exercise the complete path with duplicate messages, corrupt PDFs, misleading extensions, oversized archives, low-confidence classifications, and downstream timeouts. Reliable attachment automation is not a system that never fails. It is a system in which failures are identifiable, traceable, and recoverable without producing duplicate or unreviewed business actions.