Embeddings are the input half of every semantic search and RAG system — and the half most teams choose by vibes. This guide is the practical version: how to pick a model, embed correctly, store vectors sanely, and — most importantly — measure whether your embeddings actually work on your data.
What you're actually choosing
An embedding model maps text to a vector such that similar meaning sits nearby. The practical differences between models: dimension size (384–3,072), domain fit (general vs legal/medical/code), inference cost, and the quality of the geometry. Benchmarks can't tell you which model fits your corpus — measurement can.
The selection loop
- Start standard. Pick a mainstream model (or managed API) — don't optimize model choice before you have data.
- Build a retrieval eval set. 50–100 queries from your real usage, each with the chunks/document that should be retrieved.
- Measure hit-rate@k and MRR. The number that matters: does the right result land in the top-k?
- Compare candidates on the same eval set. Same queries, same chunks, different models — the eval set decides.
- Re-run on any model change. Embedding upgrades are regressions waiting to happen.
hit-rate@5 = fraction of queries where the known-good chunk appears in top 5
MRR = average reciprocal rank of the first good hit
Embedding correctly
The mechanics that silently break pipelines:
Normalize. L2-normalize vectors before storage — cosine similarity becomes dot product, and index math stays stable.
Batch, don't loop. Embed in batches (32–128 texts) — per-text calls multiply latency and cost.
Check the token limit. Models cap input length (usually 256–8k tokens); long documents must be chunked before embedding (chunking guide).
Keep the model version pinned. Re-embedding the corpus with a different model produces vectors in a different geometry — mixed geometries in one index are quietly broken. Version the embedding model like you version code.
Storage: vectors are just data
Vectors live wherever your data lives: a vector database at scale, or an in-memory matrix for small corpora. The storage rules:
- Store the text alongside the vector — retrieval needs the content, not just the coordinates.
- Store the model version in the index metadata — audits and re-embedding need it.
- Normalize at write time — don't trust callers to do it.
Measuring quality on your data
The eval loop in practice:
# 1. embed eval queries and candidate chunks
# 2. search each query, record ranks
# 3. compute hit-rate@k / MRR
If hit-rate@5 sits below ~70%, the fix order is: chunking first (most common culprit), then hybrid retrieval (keyword fusion), then re-ranking, then — last — a different embedding model. Re-ordering that fix list is how teams waste weeks.
The cost angle
Embedding cost scales with corpus size and re-embedding frequency. Practical controls: embed once, store forever; incremental indexing for new documents; and schedule full re-embeds only when the model version changes.
Conclusion
Embeddings are a measurement problem, not a shopping problem: start standard, normalize, pin versions, and evaluate against a retrieval eval set built from your own corpus. When retrieval underperforms, fix chunking and retrieval strategy before swapping models. The vector database guide covers the storage half; the RAG guide the pipeline half.