Define freshness as a testable contract
A vector index is a query structure derived from documents, product data, knowledge bases, or operational systems. It should not become the only source of truth. When designing refreshes, we distinguish four moments: when the source changes, when the change enters the pipeline, when its vectors are written, and when queries can retrieve them. A successful scheduled job does not by itself prove that users are seeing current information.
Not every content class needs the same freshness target. Service notices, inventory status, access controls, and incident instructions usually deserve faster propagation than archived reports. Define acceptable delay separately for additions, edits, deletions, and permission changes. If authorization metadata can become stale, do not rely solely on filters stored in the vector index; validate access against an authoritative system before returning retrieved content.
Real-time refresh turns indexing into a distributed system
A real-time path commonly begins with CDC, an outbox, a webhook, or an application event. A queue then feeds parsing, chunking, embedding, and vector upserts. The source transaction should not wait synchronously for that entire path. Persisting the event first and processing it asynchronously prevents a temporary embedding or vector database outage from blocking the primary ERP, CRM, or content-management workflow.
The hard problems are retries, ordering, and deletion. Each document needs a stable identity, a monotonically increasing source version, and idempotent writes. Consumers must discard older events and represent deletions with traceable tombstones rather than assume stale vectors will disappear when a record is absent from a later snapshot. If chunking changes, remove chunks produced by the previous document version. Otherwise, retrieval may surface both old and new passages. Real-time refresh fits frequently changing data where staleness directly disrupts work, but it requires durable infrastructure, backpressure controls, and detailed observability.
Batch refresh is controllable, but it needs more than a timer
A batch process can read a full snapshot or select incremental changes using timestamps, sequence numbers, or change tables. It works well when updates are concentrated, source systems cannot emit reliable events, or the business accepts a defined freshness window. The job should fix its query boundary when it starts, record a watermark, and advance the checkpoint only after success. Without those controls, a long-running batch may miss changes created during execution or process inconsistent ranges after a retry.
A full rebuild should not overwrite the live index in place. Build into a new namespace or generation, validate record coverage and representative queries, and then switch an alias or application setting atomically. This is especially important when changing the embedding model, vector dimensions, or chunking rules, because vectors from incompatible models should not share one search space. Batch pipelines still need resumable execution, orphan cleanup, deletion manifests, and protection against overlapping runs. Fewer scheduled executions do not eliminate consistency work.
Hybrid refresh combines a fast path with reconciliation
Many enterprise systems settle on a hybrid model: high-priority changes travel through an event pipeline, while a full or incremental batch periodically reconciles the index. New notices, revoked documents, and permission changes can take effect quickly. The batch path finds events missed during an outage, retries parsing failures, and removes vectors whose source records no longer exist. In this design, real-time processing is the speed layer and reconciliation is the correctness and repair layer.
The main hybrid risk is one path overwriting the other. A batch may read an older snapshot but finish after a newer event has already been indexed. Every indexed record should therefore carry its source version, content hash, embedding-model version, and indexing time, while writers reject version regression. If the vector database lacks conditional updates, enforce this comparison in the indexing service or an external state table. During a model migration, build a separate index generation and switch queries only after validation.
- Freshness: Determine how long each data class may be stale and which changes require priority handling before selecting technology.
- Change pattern: Understand whether traffic is steady, concentrated, or bursty, and ensure the queue can absorb source-system spikes.
- Source capability: Check for CDC, reliable webhooks, modification timestamps, or queryable change logs; source limitations often determine the viable design.
- Consistency risk: Deletions, permissions, regulated content, and operational instructions generally need stronger safeguards than ordinary reference material.
- Cost boundary: Account for parsing, embedding, vector writes, rebuilding, and retries, while avoiding re-embedding unchanged content.
- Recoverability: Retain dead-letter queues, watermarks, version history, and replayable events so missing data can be repaired precisely.
Whichever mode you choose, monitor end-to-end indexing lag, queue backlog, failures and retries, source-to-index differences, and sampled retrieval results. The practical goal is rarely zero delay for everything. It is to refresh important data quickly, process routine data efficiently, and make every missed update detectable and recoverable.