n8n

n8n Scaling: Queue Mode and Worker Configuration

Ira Menon · Jun 5, 2026 · 14 min read

Why single-instance n8n falls over

If you've been running n8n as a single container, you've probably hit the wall already. A workflow with an HTTP request that hangs for 30 seconds blocks the main process. Ten webhooks fire at once and the ninth and tenth queue up behind the first eight, adding latency nobody asked for. A long-running data transformation pegs your CPU and every other trigger in the instance slows down with it.

This happens because default n8n runs in "regular mode" — one Node.js process handles the editor UI, the REST API, webhook reception, and every single workflow execution. There's no isolation. A runaway workflow doesn't just fail on its own; it drags down everything else sharing that process.

Queue mode fixes this by splitting n8n into specialized roles: a main process that serves the UI and receives triggers, and one or more worker processes that actually execute workflows, coordinated through a Redis-backed queue. Executions get distributed across workers instead of piling up in one event loop. You can scale workers horizontally, isolate execution failures, and keep the editor responsive even when a hundred workflows are running concurrently.

This article walks through what queue mode actually changes under the hood, how to configure it with docker-compose, how to size and tune workers, and the operational details — concurrency limits, graceful shutdown, monitoring — that separate a demo setup from something you'd trust in production.

How queue mode changes the architecture

In regular mode, n8n is monolithic. One process does everything: it listens for webhook calls, renders the editor, stores execution data, and runs the workflow logic itself, synchronously, in the same event loop that's also trying to serve HTTP requests.

Queue mode decomposes this into three roles:

Main instance — serves the editor UI, exposes the REST API, and receives webhook and trigger events. When a workflow needs to run, the main instance doesn't execute it. It pushes a job onto a Redis queue and returns immediately.
Worker instances — separate processes (can be separate containers, separate machines, whatever you want) that pull jobs off the Redis queue and execute the actual workflow logic. Workers know nothing about the editor or webhook registration; their only job is "take job, run workflow, report result."
Redis — acts as the message broker (via Bull/BullMQ) between main and workers. It also backs the pub/sub channel main uses to push real-time execution status back to the editor UI, and (optionally) can serve as a cache.

The practical effect: webhook response time stops being coupled to workflow execution time. The main process acknowledges the incoming request, enqueues the job, and hands control back to the caller (or holds the connection open only if the workflow explicitly needs to respond synchronously). Meanwhile any number of workers can be chewing through the backlog in parallel.

There's also a webhook-only mode you can layer on top for very high-throughput setups — dedicated main-type instances that do nothing but receive webhooks and enqueue jobs, separate from the instance serving the editor UI. Most teams don't need this until they're past a few hundred requests per second, but it's worth knowing it exists.

Prerequisites: Redis and shared storage

Queue mode has two hard requirements beyond regular n8n:

Redis. This is non-negotiable — queue mode literally will not start without a reachable Redis instance. Use Redis 6+ in production; a single-node Redis is fine for moderate workloads, but if queue throughput matters to you, plan for Redis persistence (AOF or RDB) so an unexpected restart doesn't drop in-flight jobs.

A shared database. Regular mode is happy with SQLite. Queue mode is not — every main and worker instance needs to see the same execution data, credentials, and workflow definitions, which means you need PostgreSQL. Trying to run queue mode against SQLite (or against separate SQLite files per container) will produce inconsistent state almost immediately, since workers and main won't agree on what workflows exist or what's already run.

Here's the baseline docker-compose.yml for a queue-mode deployment with Postgres and Redis as the backing services:

version: "3.8"

services:
  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: n8n
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: n8n
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U n8n"]
      interval: 5s
      timeout: 5s
      retries: 10

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 5s
      retries: 10

  n8n-main:
    image: n8nio/n8n:latest
    restart: unless-stopped
    command: start
    ports:
      - "5678:5678"
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      - WEBHOOK_URL=https://n8n.yourdomain.com/
      - N8N_HOST=n8n.yourdomain.com
      - N8N_PROTOCOL=https
      - GENERIC_TIMEZONE=UTC
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    volumes:
      - n8n_data:/home/node/.n8n

  n8n-worker:
    image: n8nio/n8n:latest
    restart: unless-stopped
    command: worker --concurrency=10
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      - GENERIC_TIMEZONE=UTC
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    volumes:
      - n8n_data:/home/node/.n8n
    deploy:
      replicas: 3

volumes:
  postgres_data:
  redis_data:
  n8n_data:

Note the deploy.replicas block only works if you're running this through docker stack deploy with Swarm. With plain docker compose up, scale workers using docker compose up -d --scale n8n-worker=3 instead, which spins up three independent containers from the same n8n-worker service definition.

Every variable that touches queue behavior needs to be identical across main and every worker — same Redis host, same encryption key, same database. Mismatched N8N_ENCRYPTION_KEY values between main and workers is the single most common cause of "workflows fail on workers but work fine when I test them in the editor," because credentials are encrypted with that key and a worker with the wrong key can't decrypt them.

Sizing and scaling workers

The --concurrency flag on the worker command controls how many executions a single worker process handles simultaneously — not how many workers you run, but how many jobs one worker instance pulls off the queue at once. Default is 10 if you don't set it.

Concurrency and replica count solve different problems:

Increase concurrency when your workflows are I/O-bound — waiting on HTTP calls, database queries, API responses — and don't consume much CPU while waiting. A worker sitting idle on a fetch() call can easily juggle 10-20 concurrent executions without breaking a sweat.
Increase worker replicas when your workflows are CPU-bound — heavy JSON transformation, image processing, large-loop Code nodes — because concurrency within one process doesn't help if the CPU is already saturated. More processes means more CPU cores actually in play.

A reasonable starting point for a mixed workload: 3 worker replicas at concurrency 10 each, giving you headroom for 30 concurrent executions. Watch your actual queue depth and CPU/memory graphs for a week before tuning further — guessing at these numbers upfront wastes more time than just measuring.

Scaling out with docker compose:

# Scale to 5 worker replicas
docker compose up -d --scale n8n-worker=5

# Check how many worker containers are actually running
docker compose ps n8n-worker

# Watch worker logs across all replicas
docker compose logs -f n8n-worker

If you're on Kubernetes instead of docker-compose, the same logic maps onto a Deployment with a HorizontalPodAutoscaler watching CPU or a custom metric like Redis queue length:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: n8n-worker-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: n8n-worker
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Scaling on CPU alone is a decent default, but if you can wire up a custom metrics adapter that reads Bull queue depth from Redis, autoscaling on "jobs waiting" gives you a much more direct signal than CPU percentage — a workflow can be queue-bound without ever spiking CPU.

Concurrency, timeouts, and execution limits

A few environment variables matter more than people expect once you're past a handful of workflows:

EXECUTIONS_TIMEOUT — the default max runtime for a single workflow execution, in seconds. Set a sane global ceiling (e.g. 3600 for one hour) so a stuck workflow doesn't occupy a worker slot indefinitely.
EXECUTIONS_TIMEOUT_MAX — the hard ceiling that individual workflows cannot override via their own settings. Without this, someone can set a workflow-level timeout of "never" and quietly starve your worker pool.
N8N_PAYLOAD_SIZE_MAX — max request body size in MB. Bump this if you're passing large files or big JSON blobs through webhooks, but understand every increase raises worker memory pressure per execution.
QUEUE_HEALTH_CHECK_ACTIVE — enables an HTTP health endpoint on worker processes, which you'll want for container orchestration health checks (see below).

Here's an extended worker environment block with these applied:

  n8n-worker:
    image: n8nio/n8n:latest
    restart: unless-stopped
    command: worker --concurrency=10
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      - EXECUTIONS_TIMEOUT=3600
      - EXECUTIONS_TIMEOUT_MAX=7200
      - N8N_PAYLOAD_SIZE_MAX=64
      - QUEUE_HEALTH_CHECK_ACTIVE=true
      - QUEUE_WORKER_LOCK_DURATION=30000
      - QUEUE_WORKER_LOCK_RENEW_TIME=15000
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://localhost:5678/healthz"]
      interval: 30s
      timeout: 10s
      retries: 3

QUEUE_WORKER_LOCK_DURATION and QUEUE_WORKER_LOCK_RENEW_TIME control how long a worker holds an exclusive lock on a job it's processing, and how often it renews that lock. If a worker crashes mid-execution without releasing its lock, another worker won't be able to pick up that job until the lock expires — tightening these values gets stuck jobs recovered faster, at the cost of slightly more Redis chatter.

Execution data pruning

Queue mode doesn't reduce how much execution history piles up in Postgres — if anything, more throughput means more rows. Every execution, successful or failed, writes a row to the execution_entity table by default, and on a busy instance that table becomes the first thing to blow past disk expectations.

Set pruning explicitly rather than relying on defaults:

    environment:
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=336
      - EXECUTIONS_DATA_PRUNE_MAX_COUNT=50000

EXECUTIONS_DATA_MAX_AGE is in hours (336 = 14 days here). EXECUTIONS_DATA_PRUNE_MAX_COUNT caps total stored executions regardless of age, which matters more than people expect on high-volume instances — 14 days of history on a workflow firing every few seconds can still be hundreds of thousands of rows. Pruning runs on the main instance, not on workers, so this variable only needs to be set there.

Graceful shutdown and zero-downtime deploys

One thing that catches teams off guard: killing a worker container mid-execution doesn't just lose that one job — depending on your lock settings, it can leave the job in limbo until the lock times out, delaying recovery.

n8n workers respond to SIGTERM by finishing in-flight executions before exiting, up to a grace period. Make sure your orchestrator actually gives them that time instead of sending SIGKILL on a short timeout:

  n8n-worker:
    image: n8nio/n8n:latest
    command: worker --concurrency=10
    stop_grace_period: 60s
    stop_signal: SIGTERM
    environment:
      - EXECUTIONS_MODE=queue
      # ...rest of config

For rolling deploys, the pattern that works reliably:

Bring up new worker containers alongside the old ones (don't tear down first).
Let the new workers register and start pulling jobs from the queue.
Send SIGTERM to old workers and let stop_grace_period cover their in-flight jobs.
Only after old workers exit cleanly, remove them.

# Roll workers without downtime: scale up new, then old ones drain and stop
docker compose up -d --scale n8n-worker=6 --no-recreate
sleep 60
docker compose up -d --scale n8n-worker=3

This isn't elegant, but it's dependable, and it avoids the failure mode where a naive docker compose restart kills workers that still have jobs in flight.

Monitoring queue health

Once main and workers are separate processes talking through Redis, "is n8n healthy" becomes a compound question: is main up, are workers up, is Redis reachable, and — the one people forget — is the queue actually draining or just growing.

Things worth watching in production:

Redis queue length — the number of waiting jobs in the Bull queue. A queue that keeps growing over time means you're under-provisioned on workers relative to incoming trigger volume.
Worker health endpoint — with QUEUE_HEALTH_CHECK_ACTIVE=true, each worker exposes /healthz that your orchestrator's health check can poll directly, rather than inferring health from container uptime alone.
Postgres connection count — every main and worker instance holds its own connection pool. Scale workers aggressively without raising Postgres max_connections and you'll eventually get connection refusals instead of the throughput gain you wanted.
Failed job rate — Bull retains failed jobs in Redis (subject to your retry settings), and a spike here usually points at a credential problem or an external API rate-limiting you, not a queue mode issue per se.

A minimal check using redis-cli to inspect queue depth directly, useful for a cron-based alert or a quick manual check:

# Count jobs waiting to be picked up by a worker
redis-cli -h redis LLEN bull:jobs:wait

# Count jobs currently being processed
redis-cli -h redis LLEN bull:jobs:active

# Count failed jobs retained in the queue
redis-cli -h redis LLEN bull:jobs:failed

If bull:jobs:wait is consistently non-zero and climbing across several polling intervals, that's your signal to add worker replicas before users start noticing delayed executions.

Separating webhook processing from the editor

Once a queue-mode setup grows past a moderate number of concurrent webhook calls, the main instance itself can become a bottleneck — not for execution (workers handle that), but for the simple act of accepting HTTP requests, validating them, and pushing jobs onto Redis. n8n supports running additional "main-type" instances dedicated purely to webhook reception, separate from the instance serving the editor UI.

The pattern looks like this: one main instance serves /rest and the editor for your team, and one or more additional main instances (started with start --webhook-only style flags or EXECUTIONS_PROCESS variants depending on your n8n version) sit behind a load balancer and do nothing but accept incoming webhook traffic, immediately enqueueing jobs and returning.

  n8n-webhook:
    image: n8nio/n8n:latest
    restart: unless-stopped
    command: webhook
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      - WEBHOOK_URL=https://n8n.yourdomain.com/
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    deploy:
      replicas: 2

Behind an nginx or Traefik load balancer, you'd route /webhook/* and /webhook-test/* paths to this pool, and leave editor traffic on the primary main instance. Most teams don't need this split until they're consistently handling more than a few hundred requests per second, or until webhook latency starts creeping up even with plenty of worker capacity free — a sign that the bottleneck is request acceptance, not execution.

It's worth being honest about the cost here too: every additional instance type is one more thing that can silently drift out of config sync, one more thing to include in version upgrades, and one more moving part your on-call person needs to understand at 2am. Don't reach for webhook-only instances as a first optimization — reach for worker concurrency and replica count first, and only split webhook handling out once you've measured that it's genuinely the constraint.

Common pitfalls when moving to queue mode

A few mistakes show up repeatedly when teams migrate from regular mode:

Forgetting webhook nodes need `WEBHOOK_URL` set correctly on main. Workers never register webhooks — only main does — so if WEBHOOK_URL points at the wrong host, external services will fail to reach your instance even though queue processing itself is fine.
Running SQLite anywhere in the cluster. Even one instance still pointed at SQLite because someone forgot to update its environment will produce workflows that "exist" on one node and not another.
Mismatched n8n versions between main and workers. Job payload formats can change between versions; keep main and all workers on the exact same image tag, always.
No Redis persistence. An unpersisted Redis that restarts mid-burst silently drops queued jobs with no error surfaced to the user who triggered them.
Sizing workers by replica count alone, ignoring concurrency. Five workers at concurrency 1 behaves very differently than one worker at concurrency 5, even though both technically allow "5 concurrent executions" — the former gives you process-level fault isolation, the latter doesn't.

Wrapping up

Queue mode is what turns n8n from "a workflow tool that's great until it isn't" into infrastructure you can actually put load-bearing automation behind. The core idea is simple — separate triggering from execution, coordinate through Redis, scale workers independently of the editor — but the details around concurrency tuning, lock durations, pruning, and graceful shutdown are what determine whether your setup degrades gracefully under load or falls over at 2am.

Start with Postgres and Redis as non-negotiable dependencies, get a basic main/worker split running with docker-compose, then tune concurrency and replica count against your actual workload rather than a guess. Add health checks and queue-depth monitoring before you need them, not after an incident forces the issue.

If you're building serious automation pipelines — especially ones that call out to LLMs, chain agent calls, or need to survive real production traffic — this operational layer is exactly where a lot of AI-workflow projects quietly fail even when the workflow logic itself is correct. We cover this end-to-end, including queue mode, worker sizing, and deploying resilient n8n-based agent pipelines, in the n8n AI Agent Tutorial course on teachyou.ai.