A classic database stores rows and columns: customers, orders, dates. It finds information when you give it an exact criterion. A vector database stores something else: embeddings, series of numbers that represent the meaning of passages of text. It finds the passages whose meaning is closest to a question, even if the words differ.
What a vector database is for
In an AI assistant that searches passages by meaning, the vector database acts as the index. It works in three steps:
- Loading: each document in the knowledge base is split into passages, each passage gets its embedding, and the whole set is stored in the vector database with a reference to the original document
- Searching: the visitor's question in turn gets an embedding, and the database returns the passages whose vectors are closest
- Answering: these passages are passed to the language model, which writes the answer and can cite the original document
These three steps form the core of RAG, retrieval-augmented generation. The vector database makes the second step fast, even with hundreds of thousands of passages.
How it finds nearby passages
Comparing a question with every passage, one by one, would become slow at large volume. Vector databases use approximate nearest neighbour indexes, which organize the vectors so the closest ones can be found very quickly without scanning everything. The result is not always the absolute closest, but it is close enough for the purpose, with a large time saving.
Closeness is usually measured by cosine similarity, which compares the direction of two vectors. Two passages about the same topic point in similar directions, whatever their length.
Existing solutions
There are two main families of tools:
- Specialized databases, designed only for vectors, such as Pinecone, Weaviate, Qdrant or Milvus
- Extensions of classic databases, such as pgvector for PostgreSQL, which add vector search to a database already used for the rest of the application
For an application of reasonable size, extending an existing database is often enough and avoids multiplying the systems to maintain. Specialized databases make sense for very large volumes or advanced search needs.
What a vector database does not do
- It does not check facts. It finds passages with similar meaning; it does not know whether one of them correctly answers the question or is up to date
- It does not replace sorting documents. An old price list stored in the database will be found as easily as the new one
- It does not handle exact terms on its own. An order number, a product reference or a postcode is better found by a classic search. Many systems combine both, which is called hybrid search
- It is not a conversation memory. It indexes documents, not what visitors said, unless you explicitly choose to store that there
Vector databases and personal data
An embedding does not let you easily read back the original text, but it is derived from it and stays linked to it in the database. If a document contains personal data, its passages and vectors indirectly contain it too. The rules of the GDPR therefore apply: deleting a document must delete its passages and vectors, and the hosting of the database is part of the processing chain to document.
Does an SME assistant need a vector database?
Not necessarily. A vector database becomes useful when the documentation is too large to be read in full with every question. With the large context windows of current models, an SME whose knowledge base fits in a few dozen documents can often pass them to the model in full. This approach avoids passages that the search misses or that are cut off from their context, at the cost of more tokens per question.
KernChat made that choice: the sources of the knowledge spaces go into the model's context in full, in priority order, and a ranking by words shared with the question decides between sources when they do not all fit. For an organization whose documentation grew substantially, meaning-based search backed by a vector database remains a possible evolution.