Backing Up and Restoring Vector Databases
Losing a vector database is worse than losing a normal database backup, because the index behind it can take hours to rebuild from raw embeddings. A solid vector database backup strategy needs to cover three things at once: the vectors themselves, the metadata attached to each point, and the index configuration (distance metric, HNSW parameters, quantization settings) needed to recreate search behavior exactly. This guide walks through backup and restore for the vector databases teams actually run in production: Qdrant, Weaviate, Milvus, pgvector, and Pinecone, plus a portable export format you can use to move between them.
Why vector database backup is different from normal backups
A relational database backup is mostly about durability: you dump rows, you restore rows, done. A vector database backup has to preserve more:
- The raw vectors. These are usually generated by an embedding model call that costs money and time. If you lose them and don't have the source text cached separately, you are re-embedding your entire corpus.
- The index structure. HNSW graphs, IVF clusters, and product quantization codebooks are expensive to build. Some tools let you restore a raw snapshot of the index (fast), others force a rebuild from vectors (slow but portable).
- Metadata and payload filters. Most real workloads filter by metadata (tenant ID, document type, timestamp) alongside the similarity search. If metadata and vectors drift out of sync during a restore, filtered queries silently return wrong results.
- Consistency with the source of truth. Vector databases are almost always a derived index sitting next to a primary store (Postgres, S3, a document store). A backup strategy has to answer: is the vector DB the source of truth, or can it be rebuilt from source documents plus an embedding pipeline? That answer changes how urgent backups are.
Before picking a backup approach, decide which category you're in. If vectors can be regenerated deterministically from source text and a pinned model version, your backup priority is the metadata mapping, not the vectors. If vectors come from a model you no longer have access to, or from a stochastic pipeline (dedup, chunking heuristics, manual curation), the vectors themselves are irreplaceable and need real backups.
Qdrant: snapshot-based vector database backup
Qdrant has first-class snapshot support built into the API, which makes it the simplest of the group to back up correctly.
curl -X POST 'http://localhost:6333/collections/documents/snapshots'This creates a snapshot file on the server's snapshot directory. List and download it:
curl 'http://localhost:6333/collections/documents/snapshots'
curl -o documents-backup.snapshot \
'http://localhost:6333/collections/documents/snapshots/documents-2026-07-10-12-00-00.snapshot'Restore into a new or existing collection:
curl -X PUT 'http://localhost:6333/collections/documents/snapshots/upload' \
-H 'Content-Type: multipart/form-data' \
-F 'snapshot=@documents-backup.snapshot'For automated backups, wrap this in a script that runs on a schedule and ships the snapshot to object storage:
import requests
import datetime
import subprocess
QDRANT_URL = "http://localhost:6333"
COLLECTION = "documents"
def backup_qdrant():
resp = requests.post(f"{QDRANT_URL}/collections/{COLLECTION}/snapshots")
resp.raise_for_status()
snapshot_name = resp.json()["result"]["name"]
download = requests.get(
f"{QDRANT_URL}/collections/{COLLECTION}/snapshots/{snapshot_name}",
stream=True,
)
local_path = f"/backups/{snapshot_name}"
with open(local_path, "wb") as f:
for chunk in download.iter_content(chunk_size=8192):
f.write(chunk)
subprocess.run(["aws", "s3", "cp", local_path, f"s3://my-backups/qdrant/{snapshot_name}"], check=True)
print(f"backed up {snapshot_name}")
if __name__ == "__main__":
backup_qdrant()Qdrant also supports full storage-level snapshots for the whole node (not just one collection), useful when you run a single-tenant cluster and want a full disaster recovery image:
curl -X POST 'http://localhost:6333/snapshots'Qdrant's HNSW index is included in the snapshot, so restore is fast and doesn't require re-indexing. That is the main reason to prefer Qdrant's native snapshot API over a manual point-by-point export when the collection is large.
Weaviate: backup modules and restore
Weaviate ships backup as a first-class module system, with backends for filesystem, S3, GCS, and Azure blob storage. You enable a backend in the server config, then trigger backups through the API.
docker run -d \
-p 8080:8080 \
-e BACKUP_S3_BUCKET=my-weaviate-backups \
-e BACKUP_S3_ENDPOINT=s3.amazonaws.com \
-e ENABLE_MODULES=backup-s3 \
semitechnologies/weaviate:latestTrigger a backup across one or more collections:
import weaviate
client = weaviate.connect_to_local()
result = client.backup.create(
backup_id="documents-backup-2026-07-10",
backend="s3",
include_collections=["Document"],
wait_for_completion=True,
)
print(result.status)
client.close()Restore into a fresh cluster with the same backup ID:
import weaviate
client = weaviate.connect_to_local()
result = client.backup.restore(
backup_id="documents-backup-2026-07-10",
backend="s3",
include_collections=["Document"],
wait_for_completion=True,
)
print(result.status)
client.close()Two things to watch with Weaviate restores: the target cluster must have the same number of shards as the source (or you need to explicitly remap shard count), and restoring while writes are happening to the same collection name will fail, so schedule restores during a maintenance window.
Milvus: collection backups with milvus-backup
Milvus does not bundle a backup tool inside the core server; instead there's a companion CLI called milvus-backup that talks to the Milvus API and the underlying object storage (usually MinIO or S3) directly.
milvus-backup create -n documents_backup_20260710 -c documentsList backups and inspect one:
milvus-backup list
milvus-backup get -n documents_backup_20260710Restore into the same or a different Milvus deployment:
milvus-backup restore -n documents_backup_20260710 -c documents_restoredBecause milvus-backup operates at the segment level in object storage, it is efficient even for collections with hundreds of millions of vectors, since it mostly copies existing segment files rather than re-serializing every vector. If you're running Milvus on Kubernetes, pair this with a CronJob so backups happen without manual intervention:
apiVersion: batch/v1
kind: CronJob
metadata:
name: milvus-backup-nightly
spec:
schedule: "0 2 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: milvus-backup
image: milvusdb/milvus-backup:latest
command: ["milvus-backup", "create", "-n", "nightly-$(date +%Y%m%d)", "-c", "documents"]
restartPolicy: OnFailurepgvector: backup with standard Postgres tooling
If your vectors live in Postgres via the pgvector extension, the good news is that you already have a mature backup story: pg_dump, pg_basebackup, and continuous WAL archiving all work unchanged, because vectors are just another column type.
pg_dump -h localhost -U postgres -d embeddings_db \
--format=custom \
--file=embeddings_backup.dumpRestore:
pg_restore -h localhost -U postgres -d embeddings_db_restored \
--clean --if-exists \
embeddings_backup.dumpFor large tables, dump only the relevant schema and use parallel jobs to speed things up:
pg_dump -h localhost -U postgres -d embeddings_db \
--format=directory \
--jobs=4 \
--file=embeddings_backup_dirThe one pgvector-specific detail: the HNSW or IVFFlat index itself is not stored in the dump in a way that skips rebuild time. pg_restore recreates the index from scratch after loading rows, so a table with 10 million 1536-dimension vectors can take a meaningful amount of time to reindex. If restore speed matters more than storage cost, consider pg_basebackup for a full physical backup instead, which preserves the index files as-is and restores near-instantly:
pg_basebackup -h localhost -U postgres -D /backups/pg_base_20260710 -Fp -Xs -PEnable continuous WAL archiving if you need point-in-time recovery rather than just periodic snapshots:
archive_mode = on
archive_command = 'test ! -f /wal_archive/%f && cp %p /wal_archive/%f'Pinecone: export via API since there's no server to snapshot
Pinecone is fully managed, so there's no filesystem snapshot to take. Backup means exporting vectors and metadata through the API and storing them somewhere you control. Pinecone added a native backup and restore capability for pods-based indexes; check whether your index type supports it before rolling your own export.
Native backup (where supported):
from pinecone import Pinecone
pc = Pinecone(api_key="YOUR_API_KEY")
backup = pc.create_backup(
index_name="documents",
backup_name="documents-backup-20260710",
)
print(backup.status)Restore from that backup into a new index:
pc.create_index_from_backup(
backup_name="documents-backup-20260710",
index_name="documents-restored",
)If your index doesn't support native backup, fall back to a manual export using pagination over the list and fetch endpoints:
from pinecone import Pinecone
import json
pc = Pinecone(api_key="YOUR_API_KEY")
index = pc.Index("documents")
with open("pinecone_export.jsonl", "w") as out:
for ids in index.list(namespace="default"):
vectors = index.fetch(ids=ids, namespace="default")
for vec_id, record in vectors.vectors.items():
out.write(json.dumps({
"id": vec_id,
"values": record.values,
"metadata": record.metadata,
}) + "\n")Restore by reading the JSONL back and upserting in batches:
import json
def batched(iterable, size):
batch = []
for item in iterable:
batch.append(item)
if len(batch) == size:
yield batch
batch = []
if batch:
yield batch
with open("pinecone_export.jsonl") as f:
records = (json.loads(line) for line in f)
for batch in batched(records, 100):
upserts = [(r["id"], r["values"], r["metadata"]) for r in batch]
index.upsert(vectors=upserts, namespace="default")A portable export format across vector databases
If you want backups that aren't locked to one vendor's snapshot format, standardize on a plain JSONL export: one JSON object per line, with id, vector, and metadata fields. Every vector database client can produce and consume this shape, which makes it the safest long-term archival format even if it's slower to restore than a native snapshot.
import json
def export_to_jsonl(records, path):
with open(path, "w") as f:
for record in records:
f.write(json.dumps({
"id": record["id"],
"vector": record["vector"],
"metadata": record.get("metadata", {}),
}) + "\n")
def import_from_jsonl(path):
with open(path) as f:
for line in f:
yield json.loads(line)Compress these exports before shipping to object storage; embedding vectors are floats and compress reasonably well with standard gzip, and dramatically better with a columnar format like Parquet if you're exporting at scale:
import pandas as pd
df = pd.read_json("pinecone_export.jsonl", lines=True)
df.to_parquet("pinecone_export.parquet", compression="zstd")Testing restores, not just taking backups
A backup you have never restored is a guess, not a backup. Build a restore drill into your routine, not just an incident response step:
- Schedule a monthly restore test into an isolated namespace or throwaway cluster.
- Run a fixed query set against the restored index and compare results (IDs and scores) to a saved baseline from before the backup.
- Check metadata filters specifically, since a restore that gets vectors right but drops or corrupts metadata will pass a naive smoke test and fail in production.
- Time the restore. Know your actual recovery time, not the theoretical one, so you can set a realistic recovery time objective.
def verify_restore(index, baseline_results):
for query_vector, expected_ids in baseline_results:
result = index.query(vector=query_vector, top_k=10)
actual_ids = [match.id for match in result.matches]
overlap = len(set(actual_ids) & set(expected_ids))
if overlap < len(expected_ids) * 0.9:
raise ValueError(f"restore verification failed: only {overlap} of {len(expected_ids)} matched")
print("restore verified against baseline")Disaster recovery patterns worth adopting
- Keep the embedding pipeline reproducible. Pin the embedding model version and chunking logic in version control. If vectors are ever lost entirely, a reproducible pipeline turns a data-loss incident into a re-processing job instead of a permanent loss.
- Store source documents separately from the vector index. Object storage for raw text or PDFs is cheap and durable. Treat the vector database as a derived, rebuildable artifact whenever possible, and reserve expensive vector-level backups for cases where the embedding process is not fully deterministic.
- Replicate across regions for anything customer-facing. Qdrant, Weaviate, and Milvus all support multi-node clustering with replication; use it instead of relying purely on periodic backups when uptime matters.
- Version your backups, not just your data. Tag each backup with the embedding model version, index configuration, and schema version. A restore that mixes a v1 embedding model's vectors with v2 metadata will silently degrade search quality.
- Automate the backup, not just the trigger. A cron job that calls the snapshot API is not a backup strategy until it also verifies the snapshot completed, checks the file size against expectations, and alerts on failure.
FAQ
How often should I back up a vector database? Match backup frequency to how expensive it is to rebuild the index and how often the underlying data changes. For a corpus that updates daily, nightly snapshots are usually enough. For a corpus with frequent writes and strict recovery point objectives, use continuous replication (Postgres WAL archiving for pgvector, multi-node replication for Qdrant or Milvus) instead of periodic snapshots.
Do I need to back up the vector index or just the raw vectors? It depends on restore time tolerance. Raw vectors plus metadata are enough to rebuild an index from scratch, but rebuilding HNSW or IVF indexes on large collections can take hours. If fast recovery matters, back up the index snapshot itself (Qdrant snapshots, Milvus segment backups, Postgres physical backups) rather than only the raw vectors.
Can I migrate a backup from one vector database to another? Only through a portable format like JSONL, since native snapshot formats (Qdrant snapshot files, Milvus segments) are vendor-specific and not cross-compatible. Export to id, vector, metadata JSONL, then write a small import script for the target database's SDK.
What happens to filters and payload indexes during a restore? Most vector databases store payload indexes (used to speed up metadata filtering) separately from the vector index, and some restore paths skip rebuilding them for speed. After any restore, explicitly check that filtered queries return expected results, not just unfiltered similarity search, since a missing payload index degrades filter performance without causing an obvious error.
Is a database dump enough, or do I also need to back up the embedding pipeline? A database dump alone is enough only if you never need to regenerate a single vector. In practice, keep the embedding pipeline (model version, preprocessing, chunking) versioned separately so that if a backup is incomplete or the format becomes unreadable years later, you can still reconstruct the index from source documents.
How do I back up a self-hosted Milvus cluster running on Kubernetes? Use the milvus-backup CLI as a scheduled Kubernetes CronJob rather than trying to snapshot the underlying object storage bucket directly, since milvus-backup understands segment metadata and produces a consistent, restorable backup instead of a raw file copy that may capture a collection mid-write.
BootcampA 30-day guided bootcamp: build, harden and ship a production autonomous agent from scratch.
AI AgentsUnderstand how AI agents really work: the loop, the tools, the memory, and why most agent projects fail.