Treat every upload as untrusted input
An authenticated user does not make an uploaded file trustworthy. Accounts can be compromised, a partner’s workstation can be infected, and an otherwise legitimate document can contain macros, embedded objects, or content designed to exploit a parser. A newly stored file should therefore not become immediately available to an ERP, CRM, knowledge base, email workflow, or download endpoint. Its default state should be pending inspection and inaccessible.
File extensions, browser-supplied MIME types, and original names are hints rather than security evidence. The backend should identify the format independently, enforce size and type policies, and assign an internal file ID. Preserve the original name for display if needed, but do not use it directly as a storage path. That avoids path traversal, collisions, and unsafe data appearing in logs or response headers.
Separate quarantine, scanning, and release
A robust pipeline has distinct upload, quarantine, scan, decision, and release stages. The client can upload through a short-lived signed URL into a quarantine bucket. An object-created event then enters a queue, and a worker streams or downloads the content to a scanning engine. The application should consult an authoritative status record instead of assuming that the presence of an object means the upload is complete.
Model the workflow with explicit states such as UPLOADING, PENDING_SCAN, SCANNING, CLEAN, REJECTED, and SCAN_ERROR. Only CLEAN files may enter the production store or receive a download authorization. Prefer copying an approved file into a separate clean bucket as a new object instead of merely changing a tag or permission on the quarantined object. Separate storage provides clearer IAM boundaries, retention rules, and incident response options.
- Quarantine storage: Permit access only from the upload service and scanner, never from normal applications or a CDN.
- Event queue: Absorb traffic bursts, support retries, and prevent scanner outages from blocking the upload API.
- Scan worker: Enforce limits for CPU, memory, file size, archive depth, extracted content, and execution time.
- Clean storage: Accept writes only from the release service after a successful policy decision.
- Status database: Record the verdict, engine version, file hash, timestamps, policy version, and failure reason.
An inconclusive scan is not a clean result
A scanner needs more than clean and infected outcomes. Password-protected archives, corrupted documents, archive bombs, deeply nested containers, oversized files, and timeouts can all prevent complete inspection. Classify these results as inconclusive or SCAN_ERROR rather than silently treating them as safe. High-sensitivity systems normally fail closed. Lower-risk workflows may route the file to manual review, but should still withhold it from downstream systems.
Malware scanning should also be only one control in the decision. Validate allowed formats, actual file signatures, macros, executable content, and container depth. For externally supplied PDF or Office documents, content disarm and reconstruction may be appropriate, but it can break digital signatures, forms, macros, or layout. Use it when the business purpose permits a sanitized derivative; do not apply it indiscriminately to files that must retain their original semantics.
Content hashes can prevent unnecessary rescanning, but a cache key must include the scanner engine, signature database, and policy version. A CLEAN verdict from an older definition set is not permanently valid. When threat intelligence changes, the platform should be able to identify released files that require another scan and, if necessary, suspend their download access while reassessment runs.
Choose synchronous or asynchronous scanning by workflow
Synchronous scanning can provide a simple experience for small files when the user genuinely needs an immediate result. The request path, however, then inherits scanner latency, traffic spikes, and timeout behavior. For large files, variable scan times, or bursty upload volumes, an asynchronous pipeline is usually more resilient. The interface can show a processing state and deliver the result through polling, a webhook, or a message.
The difficult part of asynchronous processing is idempotency. Object events may be delivered more than once, and a worker may finish scanning but fail before recording the result. Tie every job to an internal file ID and object version, and use conditional state transitions. This prevents duplicate releases and stops a late, stale worker from overwriting a newer decision. After bounded retries, move the job to a dead-letter queue and raise an actionable alert rather than retrying forever.
Design operations and recovery into the first version
Monitor queue depth, age of the oldest pending job, scan latency, error classes, rejection reasons, and deployed scanner versions. Alerts should distinguish malware being correctly blocked from the platform being unable to scan. The first is security telemetry; the second is a reliability incident. Audit records should answer who uploaded the file, when it was scanned, which engine and policy produced the verdict, and which service released it.
Before launch, exercise the entire path with a harmless antivirus test file, corrupted input, encrypted archives, duplicate events, scanner timeouts, and a fully unavailable scanner. Verify that ordinary application identities cannot bypass the pipeline and write directly to clean storage. Enterprise uploads often span portals, LINE workflows, ERP, CRM, and cloud storage; a shared trust boundary and status model are generally easier to operate than adding an unrelated scanner call inside every integration.