Embeddings: Turning Text into Vectors

What Embeddings Capture

Embedding models map words and sentences to fixed-size vectors while preserving meaning. Two different sentences that mean the same thing produce vectors that are close together.

Generating Embeddings Locally

sentence-transformers gives you free embedding models that run on your own machine.

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
texts = [
    "Vector databases store embeddings for fast search.",
    "An elephant uses its trunk to drink water.",
    "Semantic search matches by meaning.",
]
vectors = model.encode(texts)
print(vectors.shape)  # (3, 384)

query = model.encode(["How do vector stores work?"])[0]
print(query.shape)    # (384,)

Dimensions Matter

384-D vectors are small and fast for a laptop demo. Production API models often return 1024, 1536 or 3072 dimensions; bigger vectors capture more nuance but cost more memory and per-query compute.

Normalization

Normalizing vectors to unit length makes cosine and dot-product search equivalent, and many ANN indexes prefer it. Normalize both stored and query vectors.

Key Points

• sentence-transformers runs free local embeddings.
• The MiniLM model returns 384-dimensional vectors.
• Track dimensions: they drive index size and cost.
• Normalize vectors when using ANN indexes.