n8n

n8n for Data Synchronization Between Systems

Ira Menon · Jun 9, 2026 · 15 min read

Why Data Sync Keeps Breaking (And Why n8n Fixes It)

Every growing company hits the same wall: the CRM doesn't talk to the billing system, the support desk doesn't know about new signups, and the spreadsheet someone built in 2022 is now "the source of truth" for inventory. Data lives in silos, and someone eventually has to move it between them - manually, on a schedule, or in a panic when a customer complains that their order status is wrong in one place but not another.

This is data synchronization, and it's one of the most common - and most underestimated - problems in software. Teams either write custom scripts that break when an API changes, buy an expensive iPaaS (integration platform as a service) subscription with per-task pricing that spirals out of control, or just accept that their systems will drift out of sync and someone will "fix it later."

n8n sits in a sweet spot. It's a workflow automation tool with a visual, node-based editor, but underneath that visual layer is a full-fledged JavaScript/Python execution environment, native HTTP request handling, database connectors, and a self-hostable architecture that means you're not paying per task or per row synced. If you've ever tried to keep two systems consistent - a CRM and a data warehouse, two databases, an e-commerce platform and an accounting tool - n8n gives you the primitives to do it reliably, observably, and without vendor lock-in.

This article walks through how data synchronization actually works in n8n: the patterns, the node types you'll rely on, the pitfalls that trip up beginners, and how to think about sync architecture so your workflows don't become the next fragile system nobody wants to touch.

What "Data Synchronization" Actually Means in Practice

Before touching a single node, it helps to be precise about what you're building, because "sync" is used loosely and the implementation differs a lot depending on the answer.

One-way sync: Data flows from System A to System B only. Example: new Shopify orders get written into a PostgreSQL analytics database. B never writes back to A.
Two-way sync: Changes in either system propagate to the other. Example: updating a contact in HubSpot updates it in Airtable, and vice versa. This is significantly harder because you need to avoid infinite loops (A updates B, which triggers B updating A, which triggers A again).
Batch sync: Runs on a schedule - every 15 minutes, hourly, nightly - and processes everything that changed since the last run.
Real-time sync: Triggered immediately by an event - a webhook fires the moment a record is created or updated, and the workflow reacts within seconds.
Full sync vs. delta sync: Full sync pulls the entire dataset every time and reconciles it. Delta sync only pulls what changed (using timestamps, version numbers, or change-data-capture) since the last successful run.

Most production n8n workflows end up being one-way or two-way delta syncs, either scheduled or event-driven. Picking the wrong shape early - like building a full sync when you actually needed delta sync - is the single biggest cause of slow, expensive, and unreliable pipelines. Get this decision right before you open the n8n editor.

The Core Building Blocks You'll Use Repeatedly

n8n's strength for sync work comes from a small set of node types that combine in almost every workflow you'll build.

Trigger nodes: The Schedule Trigger (cron-based) for batch sync, and Webhook nodes for real-time sync. Many app-specific nodes (Airtable, HubSpot, Google Sheets) also ship with their own trigger variants that poll for new/updated records without you writing polling logic yourself.
App and database nodes: Pre-built connectors for Postgres, MySQL, MongoDB, Airtable, Google Sheets, Salesforce, HubSpot, Notion, and hundreds more. These abstract away authentication and pagination.
HTTP Request node: Your escape hatch for any system without a dedicated node - which, realistically, is most internal tools and niche SaaS products.
Set / Edit Fields node: Reshapes data between the source schema and the destination schema. This is where most of your actual "mapping" logic lives.
IF / Switch nodes: Branch logic - for example, routing "new record" vs. "updated record" vs. "deleted record" down different paths.
Merge node: Combines two data streams - critical for comparing "what's in System A" against "what's in System B" to compute a diff.
Code node: When declarative nodes aren't enough, drop into JavaScript or Python for custom transformation, deduplication, or comparison logic.
Error Trigger / NoOp / Wait nodes: For handling failures gracefully and controlling execution flow.

Understanding how these compose is more valuable than memorizing any single node's options, because sync workflows are really just: trigger, fetch, transform, compare, write, log.

Pattern 1: One-Way Scheduled Sync (The Workhorse)

This is the pattern you'll build most often, and it's the right starting point if you're new to sync workflows. The shape is:

Schedule Trigger fires every N minutes.
Fetch node pulls records from the source system, filtered to "updated since last run."
Transform the records into the destination's expected shape.
Upsert into the destination (insert if new, update if existing).
Log the run - what was synced, what failed, and the new "last synced" timestamp.

Here's a concrete example: syncing new and updated customers from Stripe into a Postgres table used for internal reporting.

Schedule Trigger (every 15 min)
   -> HTTP Request: GET /v1/customers?created[gte]={{lastRunTimestamp}}
   -> Set: map Stripe fields to Postgres column names
   -> Postgres node: Upsert on customer_id
   -> Set: update lastRunTimestamp in a config table

The critical detail beginners miss: where does `lastRunTimestamp` live? Don't hardcode it or rely on n8n's execution history alone. Store it in a small "sync state" table (even a single-row Postgres table or an Airtable base works) and read/write it explicitly at the start and end of every run. This makes your workflow resumable - if it crashes mid-run, you know exactly where to pick up, and you can inspect sync state without digging through execution logs.

For upserts, use the native "upsert" mode most database nodes provide, matching on a unique key (customer ID, email, external ID) rather than doing a delete-then-insert, which risks orphaned records if the workflow fails partway through.

Pattern 2: Real-Time Event-Driven Sync

When "every 15 minutes" isn't fast enough - say, you need a new signup in your auth provider to immediately create a corresponding record in your billing system - switch to webhook-driven sync.

Webhook node (receives event from System A)
   -> IF: check event.type == "user.created"
   -> Set: map payload to destination schema
   -> HTTP Request: POST to System B's API
   -> Respond to Webhook: 200 OK

A few things that matter here that don't come up in scheduled sync:

Idempotency: Webhooks get retried by the sending system if it doesn't receive a fast 200 response. Your workflow might receive the same event twice. Always upsert on an external ID rather than blindly inserting, or you'll get duplicate records.
Respond quickly: Use the "Respond to Webhook" node early in the workflow (or set the webhook to respond immediately) if the downstream processing is slow, so the sending system doesn't time out and retry unnecessarily.
Signature verification: Most platforms (Stripe, GitHub, Shopify) sign their webhook payloads. Verify the signature in a Code node before trusting the payload - this is a security requirement, not an optional nicety.
Queue mode for volume: If you expect high-throughput webhooks, run n8n in queue mode (with Redis and separate worker processes) so webhook receipt and workflow execution are decoupled. A burst of 500 events won't block your webhook endpoint from responding.

Pattern 3: Two-Way Sync Without Creating Infinite Loops

Two-way sync is where people get burned. The naive approach - "when A changes, update B; when B changes, update A" - creates a loop: A's change updates B, which triggers B's "changed" event, which updates A again, which triggers A's event, forever.

The fix is to track the origin of a change and skip re-propagating it:

Timestamp comparison: Before writing to the destination, compare "last modified" timestamps. If the destination's timestamp is newer than or equal to the source event's timestamp, skip the write - it means this update likely originated from the destination side already.
Sync metadata field: Add a field like last_synced_by or sync_source to your records. When your workflow writes a record, it also writes sync_source = "n8n". The trigger for the reverse-direction workflow checks this field and ignores events where the change came from n8n itself.
Change fingerprinting: In a Code node, hash the fields you actually care about syncing (ignoring metadata like updated_at) and compare the hash to the last-known hash for that record. If nothing meaningful changed, skip the write. This also protects you from loops caused by the destination system silently touching an "updated at" field on read.

Two-way sync is genuinely one of the harder problems in integration engineering. If you're building this for the first time, start by logging every write decision (synced / skipped / conflict) to a table so you can debug loop behavior by reading a log instead of guessing from execution history.

Handling Schema Differences and Data Transformation

Systems almost never use the same field names, data types, or structures, and this is where a huge share of your workflow-building time actually goes.

Common transformation problems you'll hit:

Field name mismatches: first_name vs firstName vs fname. Solved with the Set/Edit Fields node, mapping explicitly.
Nested vs. flat structures: One system returns { address: { city, zip } }, the other expects flat city, zip columns. Use the Set node's dot-notation support or a Code node for anything deeply nested.
Type coercion: Dates as ISO strings vs. Unix timestamps vs. MM/DD/YYYY. Use the Code node with DateTime (n8n bundles Luxon) to normalize consistently rather than string-slicing dates, which breaks the moment a format assumption changes.
Enum/value mapping: System A's status: "active" needs to become System B's status: 1. Build a small mapping object in a Code node or Set node rather than a long chain of IF nodes - it's easier to maintain as the source of truth for the mapping.
Multi-record fan-out: One source record needs to become multiple destination records (e.g., one order becomes N line-item rows). Use the "Split Out" node to explode arrays into individual items before writing.

A practical tip: build your transformation logic as a single, well-commented Code node rather than spreading it across ten Set nodes. It's easier to test, easier to read six months later, and easier to hand off to a teammate. Reserve the visual Set nodes for simple, obvious renames.

// Code node: normalize incoming CRM contact into internal schema
const item = $input.item.json;

return {
  json: {
    external_id: item.id,
    full_name: `${item.first_name} ${item.last_name}`.trim(),
    email: item.email_address?.toLowerCase(),
    status: item.status === "active" ? 1 : 0,
    synced_at: new Date().toISOString(),
    sync_source: "n8n",
  },
};

Deduplication, Conflict Resolution, and Data Integrity

Sync workflows fail silently far more often than they fail loudly, and the usual culprit is duplicate or conflicting data rather than a crashed workflow.

Deduplicate on a stable key: Never dedupe on something that can change (like an email address, if users can update theirs). Use the system's immutable external ID wherever one exists.
Use the Merge node's "Compare" mode: To build a proper diff between source and destination, pull both datasets, then use Merge in comparison mode to identify records that exist in one but not the other, or that differ on key fields. This is the pattern for building a genuine reconciliation job, not just an append-only pipe.
Last-write-wins vs. conflict flagging: Decide upfront whether the most recent write should simply overwrite the other side (simplest, but can silently lose data) or whether conflicting concurrent edits should be flagged into a review queue instead of auto-resolved. For anything touching money or contracts, flag - don't auto-resolve.
Soft deletes over hard deletes: When a record is deleted in the source system, prefer marking it deleted_at in the destination rather than actually deleting the row. This protects you from a sync bug that misinterprets "record not returned in this page" as "record was deleted," which is a classic pagination bug.

Error Handling and Observability

A sync workflow that fails silently is worse than no sync workflow, because people start trusting stale data without knowing it. Build these in from day one, not after the first incident:

Error workflows: Attach an Error Trigger workflow to every sync workflow. On failure, send a Slack message or email with the workflow name, the failing node, and the error message so a human finds out within minutes, not days.
Retry with backoff: Use the built-in "Retry On Fail" setting on HTTP Request and database nodes for transient failures (rate limits, brief network blips) instead of failing the entire run over a single timeout.
Batching for rate limits: Use the "Split In Batches" node (or the SplitInBatches/Loop Over Items node in newer n8n versions) to process records in chunks of 10-50 with a short Wait between batches, so you don't blow through an API's rate limit halfway through a 10,000-record sync.
Execution logging table: Write a row per sync run - start time, end time, records processed, records failed, and any error text - into a dedicated logging table. This turns "did the sync work last night?" from a guessing game into a two-second query.
Dead-letter pattern: When a specific record fails to transform or write (bad data, missing required field), don't let it kill the whole batch. Catch the error per-item, write the failed record and its error to a "needs review" table, and continue processing the rest.

Loop Over Items (batch size: 25)
   -> HTTP Request (Retry On Fail: 3 attempts, wait 2s)
   -> IF: request failed
        -> true: write to dead_letter_queue table
        -> false: continue to destination write
   -> Wait: 500ms

Choosing Between Native Nodes, HTTP Request, and Code

A common early mistake is either over-relying on the Code node for everything (turning n8n into "just a place to run scripts," which loses most of the visual debugging benefit) or refusing to use Code at all and fighting the visual nodes into unnatural shapes.

A reasonable default:

Use native app/database nodes whenever one exists for your source or destination - they handle pagination, auth refresh, and rate-limit headers correctly out of the box, which is easy to get subtly wrong yourself.
Use HTTP Request for anything without a native node, or when the native node doesn't expose a specific endpoint or parameter you need. Pair it with n8n's credential system rather than hardcoding API keys in the node.
Use Code for transformation logic, conditional branching that would require a dozen IF nodes, deduplication logic, and anything involving date math or hashing.
Avoid Code for things the platform already does well - don't hand-roll pagination loops in JavaScript when the native node's "Return All" option already paginates correctly.

This mixed approach keeps workflows debuggable - you can look at the canvas and understand the shape of the sync in ten seconds, then drop into the Code nodes only where the real complexity lives.

Scaling Up: Sub-Workflows, Queue Mode, and Multi-System Sync

Once you're syncing more than two systems, or handling meaningful volume, a few architectural patterns keep things manageable:

Sub-workflows for reusable logic: If three different sync workflows all need to "normalize a customer record," extract that into its own workflow and call it with the Execute Workflow node instead of duplicating the Code node three times. When the schema changes, you fix it once.
Hub-and-spoke over point-to-point: If you're syncing four or five systems, resist the urge to build direct A-to-B, B-to-C, A-to-C workflows for every pair. Instead, sync everything to and from a central store (a Postgres database or a data warehouse), and let each system sync only with the hub. This turns an unmanageable N-squared web of workflows into a manageable N.
Queue mode for production: For any sync workflow handling real business volume, run n8n with queue mode enabled (separate main and worker processes backed by Redis). This means one slow or stuck execution doesn't block webhook intake or other scheduled runs.
Environment separation: Keep a staging n8n instance (or at least staging credentials) so you can test schema changes against sandboxed APIs before they touch production data. Syncing test data into a production CRM is a mistake you only make once.

Getting Started: A Practical First Project

If you're new to this, don't start with two-way sync between five systems. Start small and build up:

Pick one source and one destination you actually use (Google Sheets and a Postgres table is a fine sandbox).
Build a scheduled, one-way, delta sync using a stored "last synced" timestamp.
Add upsert logic keyed on a stable ID.
Add an error workflow that notifies you on failure.
Add a logging table that records each run's outcome.
Only then, if you need it, layer on real-time webhooks or two-way sync with loop protection.

Each of these steps is independently useful, and building them in order means you understand exactly what's happening at each layer instead of debugging five new concepts at once when something breaks at 2 a.m.

Data synchronization isn't a one-time build - it's infrastructure you maintain. The workflows that hold up over time are the ones with clear state tracking, real error handling, and transformation logic you can actually read six months later. n8n gives you the visual clarity to reason about the flow and the code-level power to handle the messy edge cases that always show up once real data hits the pipeline.

If you want a guided, hands-on path through building these patterns - including agent-driven workflows that can make sync decisions dynamically rather than following fixed rules - check out the n8n AI Agent Tutorial course on teachyou.ai, where we build production-grade automations step by step, sync logic included.