Embeddings and Vector Search: A Beginner's Guide

Imagine being able to search for “a movie with superheroes who fly” and getting relevant results even if the database doesn’t contain exactly those words. This is possible thanks to embeddings and vector search, two pillars of modern AI that enable machines to understand meaning rather than counting words.

What is an embedding?

An embedding is a numerical representation of a text, an image, or a sound in the form of a vector in a multidimensional space. Each dimension captures a semantic characteristic: for example, the proximity between “king” and “queen” or between “cat” and “dog”.

• Similar words have vectors that are close in space.
• Arithmetic relations work: vector(king) - vector(man) + vector(woman) ≈ vector(queen).
• Embeddings can be applied to entire sentences or to images.

How do we create an embedding?

Models like Word2Vec, BERT or CLIP learn these representations from large amounts of data. During training, the neural network adjusts the vectors so that elements with similar meanings are brought closer together mathematically.

• Word2Vec uses the context of words in sentences.
• BERT captures the meaning according to the full context.
• CLIP links images and texts in the same vector space.

Vector Search Explained Simply

Instead of searching for exact keywords, vector search compares the similarity between vectors. We often use cosine distance or Euclidean distance to find the elements closest to the query vector.

• Indexing: we store all vectors in a specialized database (Pinecone, Weaviate, FAISS).
• Query: we convert the question into a vector and then search for the nearest neighbors.
• Results: we obtain semantically close documents even without any words in common.

Concrete Examples of Use

Modern chatbots use embeddings to retrieve the right answers from documentation. Netflix or Spotify's recommendation engines compare your vector profile with those of other users. Image search tools allow finding photos with a simple text description.

How to Implement a Simple Search

Here is a minimalist example in Python using the sentence-transformers and FAISS libraries:

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np

model = SentenceTransformer('all-MiniLM-L6-v2')
docs = ["Le chat dort sur le canapé", "Le chien joue dans le jardin"]
embeddings = model.encode(docs)
index = faiss.IndexFlatL2(embeddings.shape[1])
index.add(embeddings)
query = model.encode(["Un félin qui se repose"])
D, I = index.search(query, k=1)
print(docs[I[0][0]])

This code converts sentences into vectors, indexes the database, and retrieves the sentence closest to the query.

Points of Attention and Perspectives

Embeddings can reproduce biases present in the training data. Quality depends heavily on the chosen model and the size of the corpus. Despite these limitations, vector search makes AI more intuitive and accessible, paving the way for ever more performant applications.

Embeddings and vector search are no longer reserved for experts: they are becoming accessible tools that transform the way we interact with information. By understanding these concepts, you are already ready to explore concrete projects and leverage the semantic power of AI.