Taylor Moore
Quick question: What changed between contract_v1.pdf and contract_v2.pdf?
If your answer is "I'd have to diff them manually" or "I don't know," you have a document version control problem.
And that problem is costing you more than you think.
Software engineers solved version control decades ago. Every code change is tracked. Every commit is diffable. Every version is recoverable.
But documents? Most teams are still doing this:
contracts/
├── contract.pdf
├── contract_v2.pdf
├── contract_v2_final.pdf
├── contract_v2_final_REAL.pdf
├── contract_v2_final_REAL_updated.pdf
└── contract_v3_USE_THIS_ONE.pdf
When you process these for RAG, each becomes an isolated island:
This creates three critical problems.
Users ask version-aware questions constantly:
"What changed in the latest policy update?"
"Show me the differences between Q2 and Q3 reports."
"When was this clause added to the contract?"
"What did section 4.2 say in the previous version?"
Without version tracking, your RAG system is blind to document evolution. It can only answer questions about the current state, never the history.
Real Example: A legal tech company built a contract analysis system. Users constantly asked "What changed?" The system couldn't answer because documents were processed in isolation.
They had to build a separate diffing system—another 3 weeks of engineering time on top of their existing pipeline.
Consider this scenario:
Contract v1: 1,000 chunks → $10 to embed
Contract v2: One paragraph updated
Savings: 95%
Now multiply this by every document that gets updated:
Most RAG systems re-embed 50-90% redundant content. That's money burned on compute that provides zero value.
For compliance-heavy industries, document history isn't optional:
Without version tracking, these questions require manual archaeology through file systems and email threads. That's not just inefficient—it's a compliance risk.
Good document version control has three components:
When contract_v2.pdf is uploaded, the system should automatically:
contract_v1.pdfThis is what we call auto-linking. It works through:
// How Raptor auto-links documents
const signals = {
filenameSimilarity: 0.85, // "contract_v1" vs "contract_v2"
fileSizeProximity: 0.92, // Within 10% file size
uploadTimeCorrelation: 0.8, // Uploaded 2 hours apart
contentFingerprint: 0.94, // First 2KB similarity
chunkOverlap: 0.91, // 91% of chunks match
};
const confidence = calculateConfidence(signals); // 92%
if (confidence > threshold) {
autoLink(newDocument, parentDocument);
}No manual tagging. No metadata spreadsheets. The system figures it out.
Evidence: Screenshot of the Version Intelligence surface where every upload is auto-linked, and analysts can jump straight into comparing versions and processing variants with a single click.
Version control at the document level isn't enough. You need chunk-level granularity:
const diff = await raptor.compareDocuments('contract-v1-id', 'contract-v2-id');
console.log(diff.summary);
// "Contract v2 has 50 new chunks, 12 modified chunks, 3 removed chunks"
console.log(`Similarity: ${diff.similarityScore}%`); // 91%
// See exactly what changed
diff.addedChunks.forEach((chunk) => {
console.log(
`+ [Section ${chunk.sectionNumber}] ${chunk.text.substring(0, 100)}...`
);
});
diff.removedChunks.forEach((chunk) => {
console.log(
`- [Section ${chunk.sectionNumber}] ${chunk.text.substring(0, 100)}...`
);
});Now you can answer "What changed?" with precision.
Version history should be queryable like any other data:
// Get complete version history
const lineage = await raptor.getDocumentLineage('document-id');
console.log(`Found ${lineage.totalVersions} versions`);
lineage.documents.forEach((doc) => {
console.log(`${doc.versionLabel || doc.filename}`);
console.log(` Created: ${doc.createdAt}`);
console.log(` Similarity to parent: ${doc.similarityScore}%`);
console.log(` Changes: +${doc.addedChunks} -${doc.removedChunks}`);
});
// Query: "What did section 4.2 say in version 1?"
const v1Chunks = await raptor.getChunks('version-1-id', {
filter: { sectionNumber: '4.2' },
});Once you have version control, you can build version-aware retrieval:
async function handleWhatChangedQuery(documentId: string) {
const lineage = await raptor.getDocumentLineage(documentId);
if (lineage.totalVersions < 2) {
return 'This is the first version of this document.';
}
const [current, previous] = lineage.documents;
const diff = await raptor.compareDocuments(previous.id, current.id);
return {
summary: diff.summary,
addedContent: diff.addedChunks.map((c) => c.text),
removedContent: diff.removedChunks.map((c) => c.text),
similarityScore: diff.similarityScore,
};
}async function queryAtVersion(
documentId: string,
versionNumber: number,
query: string
) {
// Get specific version
const version = await raptor.getVersion(documentId, versionNumber);
// Get chunks from that version only
const chunks = await raptor.getChunks(version.variantId);
// Use these chunks for retrieval
return retrieveFromChunks(chunks, query);
}
// "What did the cancellation policy say in version 2?"
const answer = await queryAtVersion('policy-id', 2, 'cancellation policy');async function analyzeAcrossVersions(documentId: string, query: string) {
const lineage = await raptor.getDocumentLineage(documentId);
const results = await Promise.all(
lineage.documents.map(async (doc) => {
const chunks = await raptor.getChunks(doc.variantId);
const relevantChunks = retrieveFromChunks(chunks, query);
return {
version: doc.versionNumber,
date: doc.createdAt,
content: relevantChunks,
};
})
);
return results;
}
// "How has the pricing section evolved across all versions?"
const evolution = await analyzeAcrossVersions('contract-id', 'pricing terms');Version control enables intelligent deduplication. Here's how it works:
Document v1: Process 1,000 chunks
Document v2: Process 1,000 chunks (950 identical to v1)
Document v3: Process 1,000 chunks (900 identical to v2)
Total: 3,000 chunks processed
Unique content: ~1,150 chunks
Waste: 62%
Document v1: Process 1,000 chunks
Document v2: Reuse 950 chunks, process 50 new
Document v3: Reuse 900 chunks, process 100 new
Total: 1,150 chunks processed
Waste: 0%
Evidence: Actual structured-output pipeline from invoice PDFs to versioned JSON, showing how Raptor stores entity metadata for every revision so diffing, lineage, and reuse all happen automatically.
Real Numbers: Teams using Raptor's version-aware deduplication report 60-90% reduction in embedding costs. The exact savings depend on how frequently your documents update and how much changes between versions.
For each chunk in a new version, we check:
const result = await raptor.process('contract_v2.pdf');
if (result.deduplicationAvailable) {
const stats = await raptor.getDedupSummary(result.variantId);
console.log(`Chunks reused: ${stats.chunksReused}`);
console.log(`Chunks created: ${stats.chunksCreated}`);
console.log(`Cost savings: ${stats.savingsPercent}%`);
// Per-chunk dedup info
const chunks = await raptor.getChunks(result.variantId, {
includeFullMetadata: true,
});
chunks.forEach((chunk) => {
if (chunk.isReused) {
console.log(
`Chunk ${chunk.chunkIndex}: REUSED via ${chunk.dedupStrategy}`
);
} else {
console.log(`Chunk ${chunk.chunkIndex}: NEW`);
}
});
}Can you build document version control yourself? Absolutely. Here's what's involved:
Based on teams we've talked to:
| Component | Time | Complexity |
|---|---|---|
| Content fingerprinting | 1 week | Medium |
| Metadata correlation | 1 week | Medium |
| Chunk-level diffing | 2 weeks | High |
| Lineage storage | 1 week | Medium |
| Deduplication engine | 3 weeks | High |
| Query interface | 1 week | Medium |
| Testing & edge cases | 2 weeks | High |
| Total | 11 weeks |
That's assuming no major architectural changes to your existing pipeline.
Build it yourself if:
Use Raptor if:
Use a system with version control built-in from day one:
import Raptor from '@raptor-data/ts-sdk';
const raptor = new Raptor({ apiKey: process.env.RAPTOR_API_KEY });
// Upload initial version
const v1 = await raptor.process('contract.pdf', {
versionLabel: 'Initial Draft',
});
// Upload update - auto-linked automatically
const v2 = await raptor.process('contract_updated.pdf', {
versionLabel: 'Legal Review',
});
// v2 is automatically linked to v1
console.log(v2.autoLinked); // true
console.log(v2.parentDocumentId); // v1's IDYou can add version control incrementally:
We've had version control for code since 1972 (SCCS). It's 2024 and most document systems still use folder hierarchies and filename conventions.
This isn't just an inconvenience, it's a fundamental limitation that:
The good news: you don't have to build this yourself. Document version control is a solved problem. You just need to use a system that has it.
See It In Action: Upload two versions of the same document to Raptor Data. Watch the auto-linking detect the relationship and the deduplication skip redundant content.
1,000 pages/month. No credit card required.