Skip to content
Muhammet Şafak
tr
Asked by: Furkan Answered:

How do I migrate a table to PostgreSQL partitioning with zero downtime?


Question

In our SaaS app, the `audit_logs` table that holds user activity logs grows by 50M rows a month; as it grows even simple `select`s have slowed down. I want to split this table into monthly partitions on `created_at` using PostgreSQL's declarative partitioning. How do I move the live data into this new structure with zero downtime? How are indexes and foreign key dependencies affected by this?

Answer

Short answer: 50M rows a month is genuinely big — partitioning is the right move here. Use monthly RANGE declarative partitioning on created_at, and do the migration in stages so there’s no downtime.

It’s worth one sanity check first: 50M/month is a real volume, so the answer to “is your data really big or just badly modeled” is clear here — it’s really big. Partitioning is warranted.

  1. Split with native declarative RANGE. Create a new partitioned parent (PARTITION BY RANGE (created_at)) and ATTACH the monthly partitions. Unlike the old monolithic table, a query only touches the relevant month; thanks to partition pruning, simple selects get fast again.
  2. Do the migration with zero downtime. Stand up the new parent, attach the partitions, then backfill historical data in batches (e.g. 50k-row chunks, off-peak) into the partitions. Meanwhile keep the app writing: either route through a trigger/view into the new structure, or dual-write to both old and new. Once the backfill is done, swap the names in a single transaction — no outage.
  3. Indexes are created per-partition. Postgres automatically propagates the index you define on the parent to every partition — so it’s not one global B-tree but a separate index per month. That actually helps you: old months’ indexes don’t weigh down your hot data.
  4. Don’t forget the UNIQUE and FK rule. On a partitioned table a global UNIQUE/PK must include the partition key — so your PK becomes (id, created_at). FKs that reference the table are affected too; FKs to partitioned tables work in modern PG, but verify your version.
  5. Hand lifecycle to pg_partman. Use pg_partman to auto-create future months’ partitions; for retention, drop old partitions with DETACH + DROP — far cheaper than a DELETE scan, effectively instant.

Bottom line: declarative RANGE on created_at, batched backfill, the partition key in every unique key, and pg_partman for lifecycle. Build the migration around dual-write plus a single-transaction name swap and the live system never stops. Before you go this deep into DB internals, make sure your scaling need is truly “big data” — in this case it is.

Related Reading

Tags: #database#postgresql#scaling
Share:

Comments

Sign in with your GitHub account to join the discussion. Comments are stored in GitHub Discussions.

More Questions

All questions

Search the site

Start typing to search posts, projects and pages.

Esc to close Powered by Pagefind