Define the synchronization contract first
The useful question is not which CDC method is fastest, but what the downstream system must be able to trust. A warehouse may tolerate several minutes of latency but cannot silently miss deletions. A CRM integration may care more about field mapping and recoverable API retries. A real-time decision service may require low latency and transaction ordering. Without an explicit contract, teams often discover after launch that source permissions, retention settings, or recovery procedures do not match the production environment.
Document the acceptable delay, whether hard deletes must be captured, whether changes to the same entity must remain ordered, and whether the source can expose transaction logs. Clarify whether the application can be changed and whether the destination supports idempotent writes. The contract should also explain where processing resumes after an outage, how a full rebuild works, and how source-to-target consistency is verified. CDC is a continuously operated data product, not a one-time copy job.
Log-based CDC: strong coverage with explicit operational dependencies
Log-based CDC reads a database transaction log such as a binlog or WAL. It can normally observe inserts, updates, and deletes while preserving commit order, without repeatedly scanning business tables. This makes it a strong candidate for high change volumes, low-latency pipelines, and destinations that require reliable delete capture. For continuously feeding core transactional data into a data platform, search index, or secondary database, log-based capture is usually the first approach worth evaluating.
The trade-off is a tight dependency on the database engine, version, permissions, and log-retention policy. The handoff between an initial snapshot and the live stream must avoid gaps while tolerating duplicates. A source position should be persisted only after the destination commit succeeds, and logs must not expire while a consumer is behind. Table renames, type changes, and primary-key migrations may stop a connector or emit incompatible records. Before production, test database upgrades, schema changes, network interruptions, connector restarts, and recovery from an invalid or unavailable log position.
Avoid treating exactly-once delivery as the sole safety mechanism. End-to-end flows crossing databases, brokers, and APIs can still redeliver data. A more durable design records the source position, gives each change a stable identity, and makes destination writes replayable and idempotent. The pipeline can then restart from a known point without turning duplicate delivery into duplicate business state.
Timestamp polling and application events still require careful design
Timestamp polling is often the easiest method to introduce. A job queries rows whose updated_at value is beyond the last cursor, making the approach useful when log access is unavailable, data volume is moderate, and latency requirements are relaxed. A timestamp alone is not a safe cursor, however. Multiple rows can share the same value, applications may use inconsistent clocks or time zones, and a long transaction may commit after its stored update time. Use a composite cursor of timestamp plus primary key, re-read a small overlap window, and deduplicate at the destination. Ordinary polling cannot see hard deletes, so add soft-delete markers, a deletion ledger, or periodic full reconciliation.
Event-based change propagation is different: the application emits business facts such as an order being approved or two customer records being merged. This is valuable for cross-service workflows, but an event stream should not be assumed to be a complete table replica. If a database transaction commits and event publication fails, the downstream system has a permanent gap. A transactional outbox addresses that boundary by writing the business change and pending event in one database transaction, followed by a separate publisher. Events should carry a stable event ID, entity identity, version or ordering information, and schema version. Consumers must still tolerate duplicates, delays, and out-of-order arrival.
- Choose log-based CDC when low latency, hard-delete capture, and transaction order matter, and the team can manage database privileges and log retention.
- Choose timestamp polling when source access is constrained, scale is manageable, latency is flexible, and supplementary deletion and reconciliation controls are acceptable.
- Choose events when consumers need business meaning rather than row replicas and application teams can own the outbox, event versions, and consumer contracts.
- Do not compare throughput alone. Include recovery, schema evolution, replay scope, observability, and routine operating effort in the decision.
Most enterprise architectures use a deliberate combination
In practice, a mixed design is often appropriate. A core database may feed the analytics platform through log-based CDC. A legacy ERP may expose master data through timestamp polling with daily reconciliation. An outbox stream may carry business-state transitions between services. The important constraint is to prevent these pipelines from updating the same data without clear ownership. Assign an authoritative source for each entity or field and define whether conflicts are resolved by source priority, entity version, or manual review.
The production checklist should cover durable cursor or log-position storage, idempotency keys, dead-letter handling, schema compatibility, source-to-target count or hash reconciliation, and alerts for both latency and stalled progress. Prepare an executable replay procedure as well: identify the replay interval, control conflicting writes, perform the replay, and verify the result. The right CDC architecture is not the one that looks simplest during normal operation. It is the one the engineering team can explain and repair safely after outages, duplicates, schema changes, and operator mistakes.