Taylor Moore
Most RAG systems waste money. A lot of money.
We've analyzed dozens of production deployments and found a consistent pattern: 60-90% of embedding costs go toward processing content that's already been processed.
This isn't a minor inefficiency; it's the difference between a $1,000/month bill and a $10,000/month bill. At scale, it determines whether your product is financially viable.
Here's how to fix it.
Before optimizing, you need to understand your cost structure.
Total Cost = Embedding Cost + Storage Cost + Retrieval Cost + Generation Cost
For most systems:
Embedding is usually the largest single cost and the most wasteful.
Every time you embed content, you pay for:
The problem: most teams re-embed content that's already been embedded.
The Redundancy Problem
In a typical system, 60-90% of embeddings are redundant.
Impact: 50-80% cost reduction
The biggest wins come from not embedding redundant content.
For each piece of content, check if it's already been processed:
def should_embed(chunk: str, existing_chunks: dict) -> bool:
chunk_hash = hash_content(chunk)
# Exact match - don't re-embed
if chunk_hash in existing_chunks:
return False
# Fuzzy match - check similarity
for existing_hash, existing_text in existing_chunks.items():
similarity = calculate_similarity(chunk, existing_text)
if similarity > 0.95:
# Close enough to reuse
return False
return TrueBuilding this yourself requires:
Or use a system that does it automatically:
// Raptor handles deduplication automatically
const result = await raptor.process('contract_v2.pdf');
if (result.deduplicationAvailable) {
const stats = await raptor.getDedupSummary(result.variantId);
console.log(`Savings: ${stats.savingsPercent}%`);
// Typical output: "Savings: 87%"
}Impact: 30-60% cost reduction
When documents update, only process the changes.
Traditional approach:
- Contract v1: 1,000 chunks → $10.00
- Contract v2: 1,000 chunks → $10.00
- Contract v3: 1,000 chunks → $10.00
Total: $30.00
Version-aware approach:
- Contract v1: 1,000 chunks → $10.00
- Contract v2: 50 new chunks → $0.50
- Contract v3: 30 new chunks → $0.30
Total: $10.80
Savings: 64%
This requires version tracking and chunk-level diffing:
// Process new version with automatic change detection
const v2 = await raptor.process('contract_v2.pdf');
// Check what was reused
const chunks = await raptor.getChunks(v2.variantId, {
includeFullMetadata: true,
});
let reused = 0;
let created = 0;
chunks.forEach((chunk) => {
if (chunk.isReused) {
reused++;
// This chunk's embedding is reused from v1
} else {
created++;
// This chunk needed new embedding
}
});
console.log(`Reused: ${reused}, Created: ${created}`);
// Output: "Reused: 950, Created: 50"Impact: 100% cost reduction on duplicates
Identical documents should cost nothing to "process."
report.pdf and report_copy.pdfconst result = await raptor.process('document.pdf');
if (result.isDuplicate) {
console.log('Duplicate detected');
console.log(`Original: ${result.canonicalDocumentId}`);
console.log(`Processing skipped: ${result.processingSkipped}`);
console.log(`Cost: $0.00`);
// You get the chunks from the original - no reprocessing needed
}Real Impact: One team found that 23% of their uploads were duplicates. They were paying to process the same content multiple times.
With duplicate detection, those uploads became free.
Impact: 20-40% cost reduction
How you chunk affects how much you embed.
# Naive approach: fixed 500 token chunks
def chunk_fixed(text: str, size: int = 500):
tokens = tokenize(text)
return [tokens[i:i+size] for i in range(0, len(tokens), size)]This creates problems:
Chunk based on document structure:
def chunk_semantic(document):
chunks = []
for section in document.sections:
# Keep sections together
if section.token_count < max_chunk_size:
chunks.append(section)
else:
# Split large sections at paragraph boundaries
chunks.extend(split_at_paragraphs(section))
return chunksBenefits:
Use this checklist to audit your current system:
If you answered "no" to more than 3 questions, you're likely wasting 50%+ of your embedding budget.
Let's look at concrete numbers for different use cases.
Profile:
Without optimization:
Initial load: 1,000 × 3 versions × 1,000 chunks = 3M chunks
Monthly: (100 new + 200 updates) × 1,000 chunks = 300K chunks
Annual embedding cost: ~$36,000
With optimization:
Initial load: 1M unique chunks (2M deduplicated)
Monthly: 100 × 1,000 + 200 × 50 = 110K chunks
Annual embedding cost: ~$6,000
Savings: 83%
Profile:
Without optimization:
Initial load: 10,000 × 500 chunks = 5M chunks
Monthly: 500 × 500 chunks = 250K chunks
Annual embedding cost: ~$32,000
With optimization:
Initial load: 4M unique chunks (20% deduped)
Monthly: 400 × 500 = 200K chunks (20% duplicate detection)
Annual embedding cost: ~$18,000
Savings: 44%
Profile:
Without optimization:
Quarterly: 200 × 2,000 chunks = 400K chunks
Annual embedding cost: ~$19,200
With optimization:
Quarterly: 200 × 100 new chunks = 20K chunks
Annual embedding cost: ~$960
Savings: 95%
Required components:
class CostOptimizedPipeline:
def __init__(self):
self.chunk_store = ChunkHashStore() # Need to build
self.version_tracker = VersionTracker() # Need to build
self.dedup_engine = DeduplicationEngine() # Need to build
def process(self, document):
# 1. Check for duplicates
if self.is_duplicate(document):
return self.get_existing_chunks(document)
# 2. Check for version relationship
parent = self.find_parent_document(document)
# 3. Extract and chunk
chunks = self.extract_and_chunk(document)
# 4. Deduplicate chunks
new_chunks = []
reused_chunks = []
for chunk in chunks:
if self.chunk_store.exists(chunk):
reused_chunks.append(self.chunk_store.get(chunk))
elif parent and self.is_similar_to_parent_chunk(chunk, parent):
reused_chunks.append(self.get_parent_chunk(chunk, parent))
else:
new_chunks.append(chunk)
# 5. Only embed new chunks
embeddings = self.embed(new_chunks)
# 6. Store for future deduplication
self.chunk_store.store(new_chunks, embeddings)
return reused_chunks + new_chunks
# Estimated build time: 6-10 weeksimport Raptor from '@raptor-data/ts-sdk';
const raptor = new Raptor({ apiKey: process.env.RAPTOR_API_KEY });
// All optimization is automatic
const result = await raptor.process('document.pdf');
// Check your savings
if (result.deduplicationAvailable) {
const stats = await raptor.getDedupSummary(result.variantId);
console.log(`Cost savings: ${stats.savingsPercent}%`);
}
// Setup time: 30 minutes
Evidence: The Lightweight Core visualization highlights that the managed runtime is under 80KB with zero dependencies—meaning all the versioning, dedup, and savings logic lives in a portable layer instead of a sprawling DIY infrastructure project. Combined with the simple setup, Raptor Data provides unmatched ease of access to powerful document processing pipeline features.
Once you implement optimization, track these metrics:
interface CostMetrics {
// Volume
totalDocuments: number;
totalChunks: number;
uniqueChunks: number;
// Efficiency
deduplicationRate: number; // % chunks reused
duplicateDetectionRate: number; // % docs that were duplicates
versionReuseRate: number; // % chunks reused from parent versions
// Cost
theoreticalCost: number; // Without optimization
actualCost: number; // With optimization
savings: number; // Difference
savingsPercent: number;
}async function getCostMetrics(timeRange: DateRange): Promise<CostMetrics> {
const documents = await raptor.listDocuments({
createdAfter: timeRange.start,
createdBefore: timeRange.end,
});
let totalChunks = 0;
let uniqueChunks = 0;
let duplicates = 0;
for (const doc of documents) {
if (doc.isDuplicate) {
duplicates++;
continue;
}
const stats = await raptor.getDedupSummary(doc.variantId);
totalChunks += stats.chunksReused + stats.chunksCreated;
uniqueChunks += stats.chunksCreated;
}
const deduplicationRate = (totalChunks - uniqueChunks) / totalChunks;
const duplicateDetectionRate = duplicates / documents.length;
// Assuming $0.01 per 1000 chunks
const theoreticalCost = totalChunks * 0.00001;
const actualCost = uniqueChunks * 0.00001;
return {
totalDocuments: documents.length,
totalChunks,
uniqueChunks,
deduplicationRate,
duplicateDetectionRate,
versionReuseRate: /* calculate from version stats */,
theoreticalCost,
actualCost,
savings: theoreticalCost - actualCost,
savingsPercent: (1 - actualCost / theoreticalCost) * 100,
};
}Even small document sets benefit:
Check for duplicates:
Most teams are surprised by their duplicate rate.
Use a managed solution. Raptor Data includes all optimization strategies out of the box. You get the savings without the engineering investment.
True—but you don't need to re-extract and re-chunk. With proper separation of concerns, model changes only affect the embedding step, not the entire pipeline.
RAG cost optimization isn't about squeezing pennies. It's about eliminating systematic waste that can represent 60-90% of your embedding budget.
The strategies are straightforward:
The implementation is where teams struggle. Building deduplication, version tracking, and chunk lineage from scratch takes months.
The alternative: Use infrastructure that has these optimizations built-in. Get the savings without the engineering investment.
See Your Savings: Process a document with Raptor Data, then process an updated version. Watch the deduplication stats show exactly how much you'd save.
10,000 pages/month. No credit card required.