Embedding Models for Retrieval

The Embedding Layer

Retrieval quality starts with the embedding model. A model that captures meaning well retrieves the right chunks; a weak model limits everything downstream no matter how good the rest of the pipeline is.

Available Choices

• Local sentence-transformers - Free, private, runs offline; sizes from small to large.
• Cloud embedding APIs - High quality with zero infrastructure; cost per token.
• Multilingual models - Handle many languages in one vector space.

Choosing a Model

• Match the model's language to your corpus.
• Use the same model for chunks and queries.
• Dimension size drives index size and query cost.
• Benchmark on your own documents, not just leaderboards.

Version Pinning

Record the exact model name in your config, because swapping models changes the vector space. After a model change, re-embed the whole corpus before serving.

MODEL_NAME = "all-MiniLM-L6-v2"   # pinned in settings
model = SentenceTransformer(MODEL_NAME)

Key Points

• Embeddings decide your retrieval ceiling.
• Local and cloud models are both viable.
• Same model for index and query, always.
• Pin versions and re-embed on change.