Building an application often involves more than saving data in a database. As an application grows, it may also need to send messages, publish events, call other services, or trigger background jobs after something happens.
For example, imagine a simple customer support application. A user creates a support ticket, and the application saves that ticket in PostgreSQL. After the ticket is created, another part of the system needs to know about it so that it can send a notification, update a dashboard, or assign the ticket to a support agent.
At first, this sounds simple. Save the ticket and then publish an event saying that the ticket was created. The problem is that these are two separate operations. If the database operation succeeds but the event is never published, the application can end up in an inconsistent state. This is the problem that the Outbox Pattern is designed to address.
This article explains the problem and shows how to implement the basic Outbox Pattern in a Node.js application.
1. What Problem Are We Trying to Solve?
Consider what happens when a customer creates a support ticket. The application first needs to store the ticket in the database. Once the ticket has been saved, the application may also need to publish a ticket.created event so another service can process it.
A simple implementation might look like this:
const ticket = await createTicket(data);
await publishEvent({
type: "ticket.created",
ticketId: ticket.id
});
There is nothing obviously wrong with this code. The ticket is created first, and the event is published afterwards. The problem is what happens if createTicket() succeeds but publishEvent() fails. The ticket is already stored in the database, but the event was never sent. The support notification service, for example, may never know that the ticket exists.
The opposite situation can also be problematic. If the application publishes the event successfully but the database operation does not complete, another service could receive an event describing a ticket that does not actually exist.
The main problem is that the database and the message broker are two different systems. A normal database transaction can make several database operations succeed or fail together, but it cannot automatically make a database write and a message-broker operation one atomic transaction. This is commonly known as the dual-write problem.
Why Is This a Problem in Node.js?
The problem is not specific to Node.js. It can happen in almost any application that needs to update one system and notify another. Node.js applications frequently use asynchronous operations, which makes this kind of architecture common. An HTTP request might update PostgreSQL and then publish a message to a queue such as RabbitMQ, Kafka, or Amazon SQS. The application might appear to work correctly most of the time. The database update succeeds, the message is published, and the consumer receives it.
However, distributed systems are expected to deal with failures. A process can crash between the two operations. A network connection can disappear. A message broker can temporarily become unavailable. A container can be restarted at exactly the wrong moment.
For example, suppose the ticket is successfully saved and Node.js crashes before publishing the event. When the application starts again, it may have no simple way to know that the ticket was saved but its event was never published. The database contains the truth about the ticket, but the event has effectively disappeared. This is where the Outbox Pattern becomes useful.
2. What Is the Outbox Pattern?
Instead of saving the ticket and immediately publishing the event, the application stores both the ticket and an outbox record within the same database transaction. The outbox record represents a message that needs to be delivered later.
Because the ticket and the outbox record are written in the same database transaction, they succeed or fail together. If the transaction is committed, both records exist. If the transaction is rolled back, neither record exists.
A separate background process, often called an outbox worker or relay, regularly checks the outbox table for events that have not yet been published. When it finds one, it sends the event to the message broker.
This changes the problem. The application no longer needs to make the database update and message publication succeed at exactly the same time. It only needs to make the database update and creation of the outbox record atomic. The worker can then take responsibility for delivering the event.
3. Building a Small Node.js Project
To make the idea easier to understand, imagine that we are building a small support-ticket API. The project has three main responsibilities. The Node.js application accepts a new ticket and stores it in PostgreSQL. PostgreSQL also stores an outbox record describing the event that needs to be published. Finally, a background worker reads pending events from the outbox and publishes them to a message broker.
We do not need to build a complete support platform. A single ticket table and a single outbox table are enough to demonstrate the pattern. The ticket table could contain fields such as an ID, customer name, subject, and status. The outbox table needs information that allows the worker to identify and process an event. A simple version could look like this:
CREATE TABLE outbox_events ( id UUID PRIMARY KEY, event_type VARCHAR(100) NOT NULL, payload JSONB NOT NULL, published_at TIMESTAMP );
The event_type tells us what happened, while payload contains the information needed by the consumer. The published_at column can be used to determine whether the event has already been published. For a beginner project, this is enough to demonstrate the important idea.
Saving the Ticket and Event Together
The most important part of the implementation is the database transaction. When a customer creates a ticket, the application starts a transaction. It inserts the ticket into the database and then inserts the corresponding event into the outbox table. Only after both operations succeed does the application commit the transaction.
The actual Node.js code can remain quite small:
await client.query("BEGIN");
const ticket = await createTicket(client, data);
await client.query(
`INSERT INTO outbox_events (id, event_type, payload)
VALUES ($1, $2, $3)`,
[crypto.randomUUID(), "ticket.created", JSON.stringify(ticket)] );
await client.query("COMMIT");
The important part is not the exact SQL or Node.js syntax. The important part is that both database operations happen inside the same transaction. If creating the ticket fails, the transaction can be rolled back. If inserting the outbox event fails, the ticket can also be rolled back. If both operations succeed, the transaction is committed and the database now contains both the ticket and the event that needs to be delivered.
This gives the application a much stronger guarantee than simply performing two independent operations.
What Happens If Node.js Crashes?
This is where the Outbox Pattern becomes easier to appreciate. Imagine that the application receives a request to create a ticket. It saves the ticket and creates the outbox record. The transaction commits successfully, but the Node.js process crashes immediately afterwards.
The event has not yet been sent to the message broker. That sounds like a failure, but it is not the same failure we had before. The event has not been lost. It is still stored in the database. When the outbox worker runs again, it can find the pending event and attempt to publish it. The worker therefore gives the application another opportunity to deliver the event.
This is one of the main benefits of the pattern. The event becomes durable before the application attempts to deliver it to another system.
The Outbox Worker
The second part of the project is the worker. The worker’s job is deliberately simple. It looks for outbox records that have not been published, sends their messages to the message broker, and marks them as published after successful delivery.
A simplified version might look like this:
const events = await getPendingEvents();
for (const event of events) {
await publishEvent(event);
await markAsPublished(event.id);
}
The worker can run continuously in the background or execute periodically. For a beginner project, a simple polling loop is enough. For a production system, the worker would normally need additional considerations such as locking, concurrency, retries, graceful shutdown, and monitoring.
The important concept is that the API request does not need to wait for the message broker. Its responsibility is to save the business data and the event that represents the change. The worker takes care of delivery afterwards.
What If the Message Broker Is Down?
Suppose the ticket is created successfully, but the message broker is unavailable. The outbox record remains in the database.The worker attempts to publish the event, receives an error, and leaves the event available for another attempt.
This is another important difference from the original dual-write approach. The application does not lose the event simply because an external system is temporarily unavailable. A retry can happen later when the broker becomes available again.
A real application would normally keep additional information in the outbox table, such as the number of attempts and the time of the last attempt. This makes it possible to implement controlled retries rather than repeatedly trying the same operation without any tracking. For a beginner project, however, the main idea is simply this: an unpublished event remains in the outbox until it can be successfully processed.
What About Duplicate Events?
There is an important detail that the Outbox Pattern does not solve automatically. Imagine that the worker publishes an event successfully. Immediately afterwards, the worker crashes before updating the outbox record. When the worker starts again, it may see the event as unpublished and send it again. The consumer could therefore receive the same event twice.
This means that an outbox-based system usually needs idempotent consumers. An idempotent consumer is designed so that processing the same event more than once does not cause an incorrect result.
The event ID can help with this. A consumer can store the IDs of events it has already processed and ignore an event if it receives the same ID again. Although the Outbox Pattern improves event delivery, applications still need to handle the possibility of duplicate processing.
4. When Should You Use the Outbox Pattern?
The Outbox Pattern is most useful when an application needs to update a database and reliably notify another system about that change. It can be useful for applications that publish events to message queues, event streams, or other asynchronous systems. Common examples include sending notifications after a customer action, updating search indexes, starting background processing, or informing another service that data has changed.
However, not every Node.js application needs an outbox. If an application is small and does not communicate with external systems, adding an outbox may introduce unnecessary complexity. If losing an occasional notification is acceptable, a simpler approach may be enough.
The pattern becomes more valuable when losing an event could leave different parts of the application with inconsistent information.
5. Conclusion
The Outbox Pattern provides a practical way to make event-driven Node.js applications more reliable. Instead of trying to update a database and publish an event as two separate operations, the application stores the data change and its corresponding event together in the same database transaction.
A background worker can then take responsibility for publishing the event. If the message broker is temporarily unavailable or the application stops unexpectedly, the event remains stored in the database and can be processed later.
The pattern does not remove every problem associated with distributed systems. Applications still need to consider retries, duplicate events, idempotent consumers, and monitoring. However, the Outbox Pattern provides a reliable foundation for communication between a database and other services. The main idea to remember is that the outbox acts as a durable record of events that still need to be delivered.
