← Writing

Your CDC pipeline stopped. Why is your database still filling up?

Phil Borlin · October 6, 2026 · 4 min read

Suppose you pause a CDC connector while you work on the analytics database that is its destination. The source application will keep running and the analytics tables will fall behind. When you unpause the CDC connector, it is perfectly reasonable to expect the tables will catch up.

While the connector is stopped, its replication slot may keep PostgreSQL from removing write-ahead log (WAL) files the connector still needs. This keeps the data the connector needs to catch up. Meanwhile, the application keeps making changes and generating more WAL.

Those WAL files can accumulate until the source database runs out of disk space, unless a retention limit allows the required history to be removed first.

What the connector leaves behind

As the application changes data, PostgreSQL records changes in its WAL. A CDC connector can read that log to find out what changed instead of repeatedly reading the source tables.

A replication slot keeps track of the history its consumer still needs. The slot lives in the database, separately from the connector process. If the connector disconnects, the slot can continue retaining the resources needed for it to resume.

Keeping that history gives the connector a way to catch up when it restarts. It is also why stopping the process does not remove its effect on the source.

PostgreSQL can recycle or remove old WAL when it is no longer needed. A replication slot may prevent that removal, but it isn’t the only reason PostgreSQL keeps old WAL. If WAL archiving is enabled, files need to be successfully archived before they can be removed. An archiving failure can cause WAL to accumulate too. The WAL documentation talks more about this.

The longer it waits, the more history it may need

Let’s say the work on the destination takes longer than expected. The connector remains stopped, but the application continues making changes.

The amount of history that accumulates depends on the database’s activity, not just how long the connector has been off. A pause during a busy period can have a different storage cost from the same pause during a quiet one.

If space starts running low, you need to know who owns the slot and whether its consumer is expected to return. A slot left by a retired test pipeline can keep history around just like one belonging to a connector paused for maintenance. The database does not know whether its consumer is coming back.

If the consumer is gone for good, its slot is no longer serving a recovery purpose. If the consumer is only paused, removing the slot can give up history it still needs. Those situations can look similar in a list of inactive slots, but they call for different decisions.

Protecting the disk can change the recovery

You can configure how much disk space PostgreSQL is allowed to use for WAL that replication slots keep around. In PostgreSQL 18, the setting is max_slot_wal_keep_size. Its default, -1, means there’s no limit on how much disk space the WAL those slots keep around can use.

PostgreSQL checks this limit during checkpoints, which happen automatically as time passes or enough WAL is written. Until the next checkpoint, the WAL a slot holds onto can grow past the limit you’ve set. When that checkpoint runs, PostgreSQL may remove WAL your CDC connector still needs to pick up where it left off. See the setting’s documentation for more information.

Postgres may still be running normally, but our connector can no longer use that slot to catch up.

If the missing changes cannot be recovered elsewhere, we may need to copy the source tables again to rebuild the destination. A previous article explains this situation.

The slot limit helps protect us from running out of disk space, but we also need a plan for recovering if PostgreSQL removes history our connector still needs.

What you need to know before space runs out

When a connector gets paused, you should watch free disk space, WAL growth, and whether the slot still has the history the connector needs. When the connector restarts, you should check that it is catching up.

  • confirmed_flush_lsn records what the consumer has acknowledged receiving
  • restart_lsn identifies the oldest WAL the slot might still need
  • The slot’s wal_status and safe_wal_size can help show whether required history is at risk

What the consumer has acknowledged and what WAL the slot still needs are different things, so you need to watch both positions.

See the replication-slot documentation for more information.

Before you pause the pipeline, you should know the answers to three questions:

  1. Who owns the slot? Is its consumer coming back, and who can decide to remove it?
  2. When do we need to act? Will our alerts give us enough time before the disk fills or required history is lost?
  3. How will we recover? If the history is gone, how do we rebuild the destination?

Pausing the connector may be the right way to work on the destination. You still need a plan to limit and monitor the source’s disk growth, and a way to recover if PostgreSQL removes history the connector needs when it starts back up.

Sources and scope

This article focuses on persistent PostgreSQL logical replication slots, using version 18 documentation. Temporary slots, other versions, and managed-service policies can behave differently. WAL is not the only contributor to database storage growth. PostgreSQL 18 also offers an idle-slot timeout, disabled by default, that can invalidate eligible inactive slots at a checkpoint.

The broader research covered public issues from Debezium, Confluent’s JDBC connector, Apache Kafka’s Connect framework, Postgres logical decoding tools, and CDC products. It informed the questions here, not a claim about how frequently disks fill. The opening is an illustrative scenario, not a reported incident.