What is an embedding? Definition and business implications

An embedding model is a specialised neural network, distinct from the LLM, that takes a text as input and produces a fixed numerical vector as output. The dimension of this vector varies by model: 1,536 for OpenAI text-embedding-3-small, 3,072 for text-embedding-3-large, 1,024 for Voyage 3 or Mistral Embed. The higher the dimension, the more semantic nuance the vector can capture, at the cost of proportionally higher storage and compute cost. The fundamental property of an embedding is that the distance between two vectors (measured by dot product or cosine similarity) reflects the semantic proximity of the corresponding texts. “Defence lawyer” and “legal counsel” are distant in the dictionary but close in the embedding space. That is what allows, in a RAG, the retrieval of relevant passages even when the query uses different words from those of the target document.