Key Takeaways
- Embeddings capture semantic meaning, not just keywords: 'car' and 'automobile' have nearly identical embeddings despite being different words.
- They are the foundation of RAG systems: documents are embedded, stored in vector databases, and retrieved by semantic similarity.
- OpenAI's text-embedding-3-small costs $0.02 per million tokens, making embeddings extremely affordable at scale.
Choosing an Embedding Model
| Model | Dimensions | Best For |
|---|---|---|
| OpenAI text-embedding-3-small | 1536 | General purpose, cost-effective |
| OpenAI text-embedding-3-large | 3072 | Maximum accuracy, large-scale retrieval |
| Cohere embed-v3 | 1024 | Multilingual, compression support |
| BGE-large (open-source) | 1024 | Self-hosted, no API costs |
For most business RAG applications, OpenAI text-embedding-3-small offers the best balance of quality and cost. Switch to larger models only if retrieval accuracy is insufficient with the smaller model.
Document Chunking for Embeddings
Before embedding documents, you must split them into chunks. Chunk size directly impacts retrieval quality:
- Too small (50-100 words): Loses context. Retrieved chunks lack enough information to answer questions.
- Too large (1,000+ words): Dilutes relevance. Chunks contain too many topics, reducing search precision.
- Optimal (200-500 words): Balances context and specificity. Most RAG systems perform best in this range.
- Overlap: Add 50-100 word overlap between chunks to prevent information loss at boundaries.
Chunking Strategy
Use semantic chunking (splitting at paragraph or section boundaries) rather than fixed-size chunking. Documents have natural breakpoints that align with topic changes.
Frequently Asked Questions
Need Help With Implementation?
Our team specializes in implementing the solutions described in this glossary.
Get in Touch