RAG Data Governance: Incremental Sync and Consistency (Part 1)

RAG Incremental Sync Mechanism

Full‑rebuild (clearing the vector store and re‑running parsing, cleaning, chunking, embedding, and loading) works for small scales but incurs linear cost and service downtime for millions of vectors. Incremental sync processes only the data that actually changed, keeping the index in step with the source.

Three Levels of the Problem

Detect that the source has changed.

Handle the detected changes.

Keep the online service stable during processing.

Change Detection

Timestamp Polling

Periodically scan the source for rows whose updated_at field is newer than the last cursor.

Common pitfalls:

Stale timestamps trigger unnecessary re‑embedding – some systems refresh the timestamp without changing the content.

Content changes without timestamp updates – legacy code may omit updating updated_at, causing missed changes.

Clock drift – in a distributed source, nodes’ clocks differ; a write near the cursor boundary on a slower node can be skipped.

A persistent cursor that stores the maximum processed timestamp or sequence number mitigates these issues. The cursor should be persisted in a durable, decoupled store (e.g., Redis) and refreshed after each batch, not only at job end.

Content‑Hash Comparison

Compute a hash for each document or chunk; re‑processing is triggered only when the hash changes.

Granularity options:

Document‑level hash – any change forces the whole document to be re‑embedded.

Chunk‑level hash – only altered chunks are re‑embedded, dramatically reducing work for large documents with localized edits (e.g., a 50‑page doc where only page 3 changes).

Chunk‑level hashing requires a hash index table, adding query and update overhead. MD5 is discouraged for security‑sensitive deduplication; SHA‑256 offers a safer trade‑off with acceptable performance.

Change Data Capture (CDC)

CDC pushes change events as they happen, avoiding the “after‑the‑fact” nature of polling and hashing.

Log‑based CDC – reads database transaction logs (PostgreSQL WAL, MySQL binlog, Oracle redo log) and streams INSERT/UPDATE/DELETE events.

Trigger‑based CDC – adds triggers on source tables to write changes to a side table; simpler but adds write‑time overhead.

Snapshot‑diff CDC – periodically exports full snapshots and diffs them; used only when log or trigger access is unavailable, impractical for millions of rows.

Production systems usually combine mechanisms: log‑based CDC as the primary path for relational sources, supplemented by periodic hash checks; timestamp polling for object storage or API sources; and full re‑crawl with hash diff for web‑scraped data.

Consistency Issues

Single‑node vector stores have straightforward sync, but high‑availability deployments use distributed multi‑replica architectures.

Consistency Models

Strong consistency – writes wait for all replicas before returning; incurs higher write latency.

Eventual consistency – writes succeed after updating a primary replica; replicas catch up asynchronously, leading to a short window where reads may see stale data.

Causal consistency – preserves the order of causally related operations while allowing concurrent unrelated writes to be unordered.

RAG knowledge bases typically adopt eventual consistency because a few‑second to minute propagation delay is acceptable for most search queries.

Read‑Your‑Writes Problem

After a sync task writes a new chunk to the primary node, an immediate verification query may be routed to a replica that has not yet received the update, falsely indicating failure. Mitigations include routing verification reads to the primary or waiting a short, estimated replication delay before querying.

Concurrent Write Conflict Handling

High‑frequency updates can produce concurrent writes for the same source_id. Without control, the final state depends on write order. Including a source_version in upsert operations and applying updates only when the incoming version is greater than the stored version turns “last write wins” into “highest version wins.”

Different Logic for Add, Update, Delete

Add

Insertion follows the same pipeline as initial indexing (parse → clean → chunk → embed → load). Duplicate writes can occur if detection misfires or a task restarts; using upsert with a composite key source_id + source_version ensures idempotency.

Update

Any content change requires re‑embedding the affected chunk because embeddings encode the whole text segment. In vector stores like HNSW, an update translates to a delete‑plus‑insert operation; deletions may break graph navigation and trigger tombstone handling. Chunk‑level hash diffing limits re‑embedding to truly changed chunks.

Delete

Deletion is the hardest to cleanly remove from a vector index. Two strategies are common:

Soft delete – set is_deleted=true; the record is filtered out at query time but still occupies space, degrading recall over time.

Physical delete – remove the record from the index, often via a tombstone followed by an asynchronous cleanup that reconnects neighboring nodes.

Production pipelines combine soft delete for immediate non‑blocking response with periodic physical cleanup during low‑traffic windows. Deleting a source record must also locate and delete all derived chunks, which requires retaining the source_id in chunk metadata and indexing it for efficient lookup.

Conclusion

Effective RAG data governance separates change detection, handling, and service stability, chooses the appropriate detection mechanism per source type, and adopts an eventual‑consistency model with careful read‑routing and version‑based upserts to guarantee that the vector store reflects the latest source state without unnecessary rebuilds.