Embeddings

An embedding is a fixed-length list of numbers (typically 512-4096 floats) that represents the semantic content of a piece of input. The trick is that the embedding is learned: similar inputs produce similar number lists, dissimilar inputs produce different ones.

If you embed the sentences "the cat sat on the mat" and "a feline rested on the rug", the resulting vectors will be very close together even though they share almost no exact words. If you embed "the cat sat on the mat" and "quantum field theory", the vectors will be far apart. That property — meaning becomes geometry — is what makes embeddings useful.

Where embeddings show up:

• Semantic search — find documents that match a query even when the wording is different.
• RAG pipelines — retrieve relevant context before asking an LLM to generate.
• Clustering — group similar items (de-duplication, topic modeling, news clustering).
• Recommendation — find products/songs/articles similar to one a user liked.
• Classification — train a small model on embeddings instead of raw text; faster + cheaper.

The leading embedding APIs in 2026 are OpenAI's text-embedding-3-large, Voyage AI's voyage-3, and Cohere's embed-v3. Open-source leaders include nomic-embed-text and bge-large. Choice matters: a better embedding model can lift your retrieval quality more than any other single change.

A subtle gotcha: embeddings from different models are not interchangeable. If you embed your corpus with model A and your queries with model B, the geometry won't line up and recall will collapse. Always use the same model for both — and if you ever upgrade, re-embed the entire corpus.