Scaling Django Beyond Monoliths: Read Replicas and PgBouncer

The fix is usually not more services

Every few months someone asks me whether their Django application should be split into microservices because the database is struggling. Almost every time, the honest answer is no. The monolith did not run out of room. The database connection pool did.

I have watched a modest Django project — a few thousand daily users and a handful of background jobs — jump from 40 to several hundred Postgres connections after one traffic spike, mostly because every Gunicorn worker thread held a connection open and nobody had bounded the pool. CPU was fine. Memory was fine. Postgres was refusing new work because it had reached max_connections.

That is a connection problem wearing a scaling costume. The path I would take before touching the application architecture is: measure, pool the connections, offload the reads, and keep the monolith. This article covers the pieces I actually use — PgBouncer in transaction mode, PostgreSQL streaming replicas, and the Django configuration that connects them — including the parts most tutorials skip, like replica lag and read-after-write consistency.

Start with the bottleneck, not the diagram

Scaling diagrams are satisfying. They are also usually drawn before anyone checks which resource is actually exhausted. Before adding infrastructure, I want four numbers from production:

• p95 and p99 latency, not the average. Averages hide the requests that make users leave.
• Active and idle connections, grouped by state and wait event.
• Database CPU and disk wait, sampled during the slowest hour of the day.
• Slow query fingerprints from pg_stat_statements, sorted by total time rather than per-call time.

This query is usually enough to see whether you have a connection problem or a query problem:

SELECT state, wait_event_type, wait_event, count(*)
FROM pg_stat_activity
WHERE datname = 'app'
GROUP BY 1, 2, 3
ORDER BY 4 DESC;

If most sessions sit in idle, your application is holding connections it is not using — pooling will help. If sessions are active and waiting on IO or LWLock, adding a pooler will only move the queue. Fix the queries first. Indexes, removing N+1 patterns, and selecting fewer columns are still the cheapest scaling work available. I wrote about the ORM side of that in the full-stack article.

PostgreSQL connections are expensive by design

PostgreSQL uses a process per connection. Each one consumes memory, a backend slot, and scheduler attention. The default max_connections is often 100, which sounds generous until you multiply it by deployment reality:

Without a pooler, those 96 connections reach Postgres directly. With a pooler, 96 clients share 20 server connections per pool, and the database sees the second number instead of the first. That is the whole trick. Everything else is configuration discipline.

Django's CONN_MAX_AGE makes the arithmetic worse if it is set high without a pooler: every thread keeps its connection open for the duration, so a deploy that starts three new instances before the old ones drain can briefly double the connection count. Connection storms during deploys are a classic way to take down an otherwise healthy database.

What PgBouncer actually does

PgBouncer sits between Django and PostgreSQL and multiplexes many client connections onto a small number of server connections. It does not cache queries or make them faster. It decides who gets to talk to Postgres, and for how long.

Transaction mode is the usual choice because most Django requests are short transactions. transaction.atomic() and ATOMIC_REQUESTS still work: one transaction maps to one server connection for its lifetime. What breaks are features that assume session state survives between transactions — SET values, LISTEN/NOTIFY, temporary tables, and advisory locks held across requests. Django itself mostly stays inside one transaction per request, but custom middleware and raw SQL are where these assumptions sneak in.

One Django-specific detail matters: server-side cursors. With transaction pooling, a named cursor cannot survive after its transaction ends, so Django's DISABLE_SERVER_SIDE_CURSORS must be enabled. If you use psycopg 3 with server-side prepared statements, check your PgBouncer version as well; prepared statement support in transaction mode arrived in PgBouncer 1.21, and older versions need client-side preparation.

Wiring Django to PgBouncer without surprises

A minimal pooler configuration looks like this. Run one PgBouncer instance per Postgres node — one for the primary, one for each replica — because a single [databases] entry points at a single host.

[databases]
app = host=10.0.0.10 port=5432 dbname=app

[pgbouncer]
listen_addr = 0.0.0.0
listen_port = 6432
auth_type = scram-sha-256
auth_file = /etc/pgbouncer/userlist.txt
pool_mode = transaction
max_client_conn = 400
default_pool_size = 20
min_pool_size = 5
reserve_pool_size = 5
reserve_pool_timeout = 3
server_idle_timeout = 60
query_wait_timeout = 10

The Django side is ordinary configuration with three additions that matter:

import os

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "app",
        "USER": "app",
        "PASSWORD": os.environ["DB_PASSWORD"],
        "HOST": "pgbouncer-primary.internal",
        "PORT": "6432",
        "CONN_MAX_AGE": 60,
        "CONN_HEALTH_CHECKS": True,
        "DISABLE_SERVER_SIDE_CURSORS": True,
        "OPTIONS": {
            "connect_timeout": 5,
            "options": "-c statement_timeout=30000",
        },
    },
    "replica": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "app",
        "USER": "app_readonly",
        "PASSWORD": os.environ["DB_PASSWORD"],
        "HOST": "pgbouncer-replica.internal",
        "PORT": "6432",
        "CONN_MAX_AGE": 60,
        "CONN_HEALTH_CHECKS": True,
        "DISABLE_SERVER_SIDE_CURSORS": True,
        "TEST": {"MIRROR": "default"},
        "OPTIONS": {
            "connect_timeout": 5,
            "options": "-c statement_timeout=30000",
        },
    },
}

Three details are doing quiet work here. CONN_HEALTH_CHECKS makes Django verify a persistent connection before reusing it, which saves you from stale connections after a database failover. The statement_timeout stops a runaway query from occupying a pooled server connection forever. And TEST: {"MIRROR": "default"} keeps your test suite on one database instead of trying to create and migrate a replica that does not exist.

Keep CONN_MAX_AGE modest. With a pooler, it controls client-to-pooler connections, not Postgres backends, so the usual advice changes. A value between 30 and 60 seconds is a reasonable starting point; measure max_client_conn usage before raising it.

Read replicas: useful, not magic

A read replica is a standby PostgreSQL server that continuously applies the primary's write-ahead log. Reads sent to it do not compete with writes on the primary, which is exactly what a reporting page, a search endpoint, or a feed of mostly-static content needs.

What a replica does not give you: faster writes, a backup, or unlimited read scale. It is asynchronous. The primary commits, then the WAL travels, then the replica replays it. For a few milliseconds to a few seconds, the replica is answering questions about an older version of reality. Under heavy write load or network trouble, that gap can grow.

This is why I route specific endpoints to a replica rather than sending every read there automatically. Good candidates share three traits: they are read-heavy, they tolerate slightly stale data, and a wrong answer is not expensive. Analytics dashboards, article archives, search results, and recommendation feeds fit. Login checks, stock decrements, payment state, and anything a user just changed do not.

Routing reads in Django without lying to yourself

Django supports multiple databases without any extra package. Define the aliases, then either use .using("replica") explicitly or add a router that makes the decision for you. I start explicit, because automatic read/write splitting is where most read-after-write bugs are born.

# config/db_router.py
class ReplicaReadRouter:
    """Writes and migrations go to the primary. Reads opt in explicitly."""

    def db_for_write(self, model, **hints):
        return "default"

    def allow_relation(self, obj1, obj2, **hints):
        databases = {"default", "replica"}
        if obj1._state.db in databases and obj2._state.db in databases:
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        return db == "default"

Then a reporting view opts in where lag is acceptable:

from django.db.models import Count
from django.http import JsonResponse

def article_report(request):
    rows = (
        Article.objects.using("replica")
        .filter(is_published=True)
        .values("author_id")
        .annotate(total=Count("id"))
    )
    return JsonResponse(list(rows), safe=False)

If you do enable db_for_read returning "replica", remember that it applies everywhere, including the read inside a request that just wrote. Django will happily open a transaction on the replica with transaction.atomic(using="replica") if the code asks for it. The fix is not to pretend the lag is not there; the fix is to make the routing decision honest per endpoint.

The read-after-write problem, solved honestly

The moment a user saves a profile and is redirected to a page that reads from a lagging replica, they see their old data. No amount of pooling prevents this. Handle it in one of three ways:

• Keep post-write reads on the primary. Return the updated object from the write path, or render confirmation from the primary for that request.
• Use a short sticky window. Store a timestamp in the session or cache after a write and route that user's reads to the primary for a few seconds.
• Gate the replica by lag. If replay delay crosses a threshold, fail back to the primary until it catches up.

The lag check itself is one query on the replica:

SELECT now() - pg_last_xact_replay_timestamp() AS replica_lag;

Expose that number to your application through a health check or a small periodic job, and make the read decision from the measured value. A threshold around one to two seconds is reasonable for most content pages; payment and inventory logic should not depend on the replica at all.

Sizing pools with arithmetic, not vibes

Pool sizing arguments usually end with someone saying “just set it higher.” The useful rule is simpler: server connections should match what Postgres can execute concurrently, not how many clients want to talk to it.

• Client side: total worker threads × database aliases, plus deploy headroom.
• Server side: start around the number of CPU cores available to Postgres, then adjust for I/O wait. Twenty server connections per pool is a common starting point for a four-to-eight-core instance.
• Database budget: leave room under max_connections for migrations, monitoring, backups, and an admin shell. Running at 100% is not efficiency; it is a lockout waiting to happen.

Watch query_wait_timeout and PgBouncer's SHOW POOLS output. A rising wait time means the server pool is too small for the request pattern, or that long transactions are holding connections. The second cause is more common than the first, and adding connections will not fix it.

Deploying and monitoring the topology

The shape I would deploy is intentionally boring:

Django instances  ──▶  PgBouncer (primary)  ──▶  PostgreSQL primary
                  └──▶  PgBouncer (replica)  ──▶  PostgreSQL replica

Each Postgres node gets its own pooler on a private network. Run at least two PgBouncer processes behind an internal TCP load balancer so the pooler itself is not a single point of failure. Point Django only at the pooler hostnames, never directly at the database, and keep a separate direct connection available for migrations and emergency administration.

Monitor three layers: PgBouncer's SHOW POOLS and SHOW STATS, PostgreSQL's pg_stat_activity and replication lag, and your application's p95 latency. Alert on connection saturation, query wait time, and lag thresholds. Without those signals, the first symptom of a bad configuration is an outage, which is a terrible way to learn arithmetic.

A migration path you can actually follow

1. Fix the query layer first. Enable pg_stat_statements, remove N+1 patterns, and add the indexes the slow query log asks for.
2. Add PgBouncer in front of the primary. Use transaction mode, set DISABLE_SERVER_SIDE_CURSORS, and keep CONN_MAX_AGE modest.
3. Add a replica. Keep all reads on the primary until the replica is stable and lag is observable.
4. Move one read-heavy endpoint at a time. Use .using("replica"), measure, and keep the change reversible.
5. Add lag-aware fallback. Route back to the primary when replay delay crosses the threshold you agreed on.
6. Stay a monolith. Scale worker processes and the data tier horizontally before considering service extraction. Split services when team ownership or deployment cadence demands it, not because the database got busy.

When this is the wrong investment

If your application idles at 15 database connections and p95 is under 200 milliseconds, you do not need replicas or a pooler. You need to keep an eye on growth and enjoy the quiet. Adding a replica doubles the database surface you must monitor, backup, and secure. Adding PgBouncer introduces a new failure domain and a new set of configuration traps.

Also be honest about consistency requirements. A write-heavy product with strict read-after-write behavior — payments, inventory, bookings — will get less value from replicas and more complexity. In those systems, the primary is doing the work for a reason, and the right next step is usually query optimization, partitioning, or a queue, not a second opinion.

Managed platforms change the calculus. If you deploy on a provider that already includes connection pooling, use that before running your own PgBouncer. The concepts in this article still apply, but the operational burden moves to someone else, which is often the correct trade for a small team.

FAQ

Does Django support read replicas natively?

Django does not manage PostgreSQL replication, but it supports multiple databases out of the box. You define a primary and a replica in DATABASES, then route reads explicitly with .using() or automatically with a database router.

Should PgBouncer use transaction or session mode with Django?

Transaction mode suits most Django web applications because a server connection returns to the pool after each transaction. Session mode is safer for features that depend on session state, but it scales far worse. Statement mode breaks multi-statement transactions and is not suitable for typical Django apps.

Is a read replica a backup?

No. A replica receives every change from the primary, including accidental deletes, and replication lag means it is not a point-in-time copy. Keep independent, tested backups.

How do I know if replication lag is too high?

Measure replay delay on the replica with pg_last_xact_replay_timestamp() and set a threshold that matches your product — usually one to two seconds for read-heavy pages. If lag crosses the threshold, route reads back to the primary.

Can I use PgBouncer with serverless PostgreSQL?

Managed providers such as Neon and Supabase usually offer their own pooler. Use the provider pooler before adding your own PgBouncer, and check whether it runs in transaction mode, because that affects server-side cursors and prepared statements.

The goal is boring reliability

Scaling Django beyond the monolith does not have to mean splitting the application into a fleet of services. In most projects I have worked on, the real win came from acknowledging that the database was doing too many jobs with too few boundaries: connections held open, reporting queries competing with writes, and no measurement to prove any of it.

Add pooling, add a replica, route deliberately, and watch the lag. The architecture stays familiar, the deployment stays small, and the product gets room to grow. If you are working through this on your own project, my services page describes how I approach backend and database work, or you can get in touch with the specific bottleneck you are seeing.