Start with the real consistency problem
Consider an order service that writes to its database, then updates an ERP, creates a CRM activity, reserves inventory, or sends a LINE notification. If the database commit succeeds but message publication fails, the order may never reach downstream systems. Publishing first is no safer: a consumer may act on an order that the producer later fails to commit. This is the classic dual-write problem.
Distributed locks and two-phase commit are rarely a practical foundation when the participants include separate databases, SaaS products, and third-party APIs. They may not support the same protocol, and a network timeout cannot tell you whether a remote action took effect. The useful design question is therefore: which facts must commit atomically inside one boundary, which may converge later, and who owns recovery when they do not?
Outbox: reliable publication within a local boundary
The Transactional Outbox pattern fits a precise requirement: once local business state is committed, a corresponding event must eventually be published. The application writes both the domain change and an outbox row in the same database transaction. A polling publisher or change-data-capture pipeline then sends pending records to the broker. If the application stops immediately after commit, the durable outbox row remains available for another publication attempt.
Outbox does not create automatic exactly-once delivery. A publisher can send an event and crash before marking the row as processed, so duplicates are expected. Consumers need idempotency based on an event ID, business key, or entity version. When events for one aggregate must remain ordered, include an aggregate identifier and monotonic version, then preserve that ordering through partitioning, retries, and parallel consumption.
- Use it for: publishing order changes, synchronizing search indexes, or preserving audit events after a local commit.
- Do not expect it to: reverse actions already completed by several remote systems or decide the business response to a downstream rejection.
- Operate it with: consumer deduplication, bounded retries, dead-letter handling, schema versioning, and a retention policy for processed rows.
Saga: coordinating business outcomes and compensation
A Saga divides a cross-system process into independently committed local transactions. Imagine a procurement flow that creates an order, reserves a credit limit, holds inventory, and requests shipment. If shipment creation fails, the process may release inventory and credit before marking the order as cancelled. Those actions are not a database rollback. They are new business transactions with their own permissions, validation rules, and failure modes.
The feasibility of compensation should determine whether a Saga is honest about the business process. A refund is not the deletion of a payment, a notification cannot be unsent, and an ERP document already handled by an employee may require a controlled reversal. Put irreversible work as late as possible, after necessary preconditions have passed. Where automatic recovery is unsafe, model a durable pending-manual-review state instead of pretending the process can always restore its starting point.
Event choreography can work for short flows with few participants. As the chain grows, however, ownership of timeouts, branching, and recovery becomes difficult to see. A process with deadlines, alternative paths, manual intervention, or multiple compensation strategies usually benefits from an explicit orchestrator. Treat that orchestrator as a persistent state machine: the process must resume correctly after a restart rather than depend on in-memory progress.
In practice, Outbox and Saga often work together
The patterns solve different layers of the problem. A Saga decides which business step comes next and what compensation follows a failure. Outbox ensures that a service reliably announces the result of its local step. An order service can commit an order and an outbox record together; an orchestrator consumes the event and asks inventory to reserve stock; the inventory service commits its reservation and its own outbox event locally. Each message exchange may be retried, while the Saga state records the end-to-end outcome.
Choose from business invariants rather than framework preference. If local data and its event must never be separated, use Outbox. If several systems must reach a managed success, rejection, or compensated result, use a Saga. Use both when both requirements exist. Conversely, if delayed reconciliation or human approval is acceptable, a simpler state-sync process may be easier to understand and operate than a fully automated Saga.
Engineering checks before production
Consistency eventually becomes an operations concern. Commands and events should carry a trace ID, business identifier, and idempotency key. Operators need to see which step is active, the most recent error, and the next retry time. Technical failures such as timeouts may justify automatic retries; business failures such as insufficient credit should follow a defined rejection or compensation path. Applying the same unlimited retry policy to both creates stuck workflows and unnecessary downstream load.
- Inject failures before commit, after commit, after publication, and during consumption.
- Verify that every compensation is idempotent and cannot issue a second refund or release the same reservation twice.
- Provide controlled replay, reconciliation, and manual repair tools with validated state transitions.
- Monitor outbox backlog, processing latency, retry volume, dead letters, and Sagas that remain incomplete beyond their expected window.
- Evolve event contracts compatibly so deployment order does not become another hidden distributed transaction.
A robust design does not promise that every system is synchronized at every instant. It makes intermediate states visible, retryable, compensatable, and auditable. When ERP, CRM, messaging, and cloud services expose different guarantees, an experienced integration team can help translate those constraints into explicit transaction boundaries and recovery procedures.