DevPath
Tracks

Vector Databases

What a vector database adds beyond an array of embeddings: indexing, metadata filtering, upserts, and persistence. Build a tiny store and compare pgvector, dedicated stores, and FAISS.

intermediate40 min#vector-database#pgvector#faiss#metadata-filtering#upsert

Prerequisites: Vectors and Embeddings, Similarity Search and ANN Indexes

Why it matters

You can get surprisingly far with a Python list of vectors and a top_k function. Then real life arrives: documents are edited, so vectors must be replaced. Users may only see their own company's documents, so results must be filtered. The process restarts and the vectors must survive. There are fifty million of them and brute force is too slow. A vector database is the layer that handles these problems so your application code does not have to.

Intuition

Think of a library. An array of embeddings is a pile of books on the floor with a note saying which are about cooking. A vector database is the library: shelves (an index) so you do not scan every book, a catalogue with author and year (metadata) so you can ask for "cooking, after 2020", a librarian who swaps an old edition for a new one (upserts), and a building that is still there tomorrow (durability).

Crucially, the library does not change what a good book is. Similarity still comes from embeddings. The database only makes finding and managing them practical.

Source documentssource of truth
Embedding model
ANN indexvectors, ids
Metadata storeteam, lang, date
Query + filterembed the question first
Top-k results
Ingest embeds each document into an index and a metadata store. A query is embedded the same way, then the index and the filter work together to return the top-k.

How it works

What a vector store adds over a plain array:

  • An ANN index. HNSW or IVF as described in the previous lesson, so queries avoid scanning everything. Most stores let you choose the index and tune its knobs.
  • Metadata and filtering. Each vector carries a payload such as tenant, lang, created_at, or source. Queries combine similarity with conditions on that payload.
  • Upserts and deletes. Writing an id that exists replaces it. This matters because a re-embedded document must not leave a stale copy behind.
  • Persistence. Data lives on disk with a write-ahead log or snapshots, and can be restored after a crash.
  • Scale and operations. Replication, sharding, backups, access control.

Filtering is the subtle part. There are three strategies:

  1. Post-filter: run ANN for the top k, then drop non-matching results. Fast, but with a selective filter you may get fewer than k results, or none.
  2. Pre-filter: find matching ids first, then search only those. Correct, but a graph index built over everything cannot easily skip nodes, so engines either fall back to an exact scan when the filter is very selective or navigate the graph while checking the filter as they go.
  3. Filtered traversal: the search itself skips non-matching candidates, and many systems over-fetch candidates (ef larger than k) to compensate.

Good systems pick between these automatically, but you should still know that heavy filters can lower recall or raise latency.

Brute force scan
1,000,000 distance calcs
IVF, nprobe = 8
9,000 distance calcs
Illustrative work per query for n = 1,000,000 vectors. IVF with nlist = 1,000 and nprobe = 8 costs about nlist + nprobe * n / nlist = 9,000 distance calculations. The scale is logarithmic, so the real gap is over 100 times.

Where to keep vectors: principles, not products.

  • A library such as FAISS gives you fast index structures in memory and nothing else. You manage ids, metadata, persistence, updates, and concurrency. Great for research, batch jobs, and embedded use; a lot of plumbing for a multi-user service.
  • An extension to a database you already run (for example pgvector for Postgres) keeps vectors next to your relational rows. You get transactions, joins, backups, and SQL filters for free, and one system to operate. The trade-off is that very large collections or very high query rates may stress a general-purpose database more than a purpose-built engine.
  • A dedicated vector database is built around ANN: rich filtering, sharding, tuning, and often managed hosting. The trade-off is another system to run, keep in sync with your source of truth, and pay for.

A reasonable default: start with brute force or your existing database. Move to a dedicated engine when scale, latency, or filtering needs prove it, not before. Whichever you pick, keep the source of truth (the original text and ids) in your primary database, so the vector store can always be rebuilt, for example after changing embedding models.

OperationTimeNotes
Insert / upsert one vectorO(d) flat, ~O(log n * d) graph indexGraph insert must link neighbours
Delete by idO(1) flat; tombstone in graphGraphs usually mark deleted, rebuild later
Query, no filterO(n * d) flat, ~O(log n * d) ANNApproximate for ANN
Query with metadata filterDepends on selectivityPre/post/filtered traversal
Snapshot to diskO(n * d)Write every vector once

Code: a tiny in-memory vector store

We need vectors, and a real embedding model would be a network call. Instead, embed below is a small stand-in: it hashes each word into one of 32 buckets and normalises the result. It captures word overlap, not meaning, but it is deterministic and good enough to exercise the store. To use a real model, replace only the body of embed with a call to your embedding provider; everything else stays the same.

import json
import math
import re
 
DIM = 32
 
 
def embed(text):
    """Stand-in for a real embedding model: hashed bag of words, unit length."""
    vec = [0.0] * DIM
    for word in re.findall(r"[a-z0-9]+", text.lower()):
        h = 2166136261
        for ch in word:
            h = ((h ^ ord(ch)) * 16777619) % 4294967296  # FNV-1a
        vec[h % DIM] += 1.0
    n = math.sqrt(sum(x * x for x in vec)) or 1.0
    return [x / n for x in vec]
 
 
class VectorStore:
    def __init__(self):
        self.items = {}  # id -> {"vec": [...], "text": str, "meta": dict}
 
    def upsert(self, id, text, meta=None):
        self.items[id] = {"vec": embed(text), "text": text, "meta": meta or {}}
 
    def delete(self, id):
        return self.items.pop(id, None) is not None
 
    def query(self, text, k=3, where=None):
        q = embed(text)
        scored = []
        for id, item in self.items.items():
            if where and any(item["meta"].get(f) != v for f, v in where.items()):
                continue  # pre-filter: skip non-matching items
            score = sum(a * b for a, b in zip(q, item["vec"]))
            scored.append((-score, id))
        scored.sort()
        return [(id, -neg) for neg, id in scored[:k]]
 
    def dumps(self):
        return json.dumps(self.items)
 
    @staticmethod
    def loads(blob):
        store = VectorStore()
        store.items = json.loads(blob)
        return store
 
 
def show(label, results):
    print(label, " ".join(f"{id}:{score:.3f}" for id, score in results))
 
 
store = VectorStore()
store.upsert("d1", "How to reset your password", {"team": "support"})
store.upsert("d2", "Forgotten password reset by email", {"team": "support"})
store.upsert("d3", "Quarterly revenue report for finance", {"team": "finance"})
store.upsert("d4", "Pasta recipe with tomato sauce", {"team": "food"})
store.upsert("d5", "Password policy for employees", {"team": "security"})
 
q = "forgot my password reset"
show("all     ", store.query(q))
show("support ", store.query(q, where={"team": "support"}))
store.upsert("d2", "Quarterly password rotation schedule", {"team": "security"})
show("upserted", store.query(q))
store.delete("d1")
show("deleted ", store.query(q))
restored = VectorStore.loads(store.dumps())
print("restored matches:", restored.query(q) == store.query(q))

Both versions print:

all      d1:0.447 d2:0.447 d5:0.250
support  d1:0.447 d2:0.447
upserted d1:0.447 d2:0.408 d5:0.250
deleted  d2:0.408 d5:0.250 d3:0.224
restored matches: True
AppVector storeEmbedderupsert d2, new text, team=securityembed new textnew vectorok, old d2 replaced, no stale copyquery, where team=supportembed queryquery vectorscored matches only: d1
An upsert followed by a filtered query, as in the code. Because d2 now has team=security, the support filter skips it and only d1 can match.

Read the output as a story: the unfiltered query finds the password documents; the filter keeps only the support team; the upsert replaced d2 (same id, new text and metadata), so its score dropped and it left the support team; the delete removed d1; and a snapshot restored through JSON gives identical answers. The last line of the deleted result shows a weak hit, d3, only because two unrelated words happened to hash to the same bucket, a reminder that this stand-in is crude. Note that this store does an exact scan, so it is flat and correct but O(n * d) per query. A production engine would put an ANN index inside query.

Common pitfalls

  • Treating the vector store as the source of truth: keep original documents elsewhere so you can rebuild after a model change or a corrupted index.
  • Forgetting upsert semantics: adding with a fresh id on every edit leaves old versions that still match queries. Use stable ids such as doc_id#chunk_no.
  • Over-filtering with post-filter search: asking for 5 results and then discarding 4 of them for permissions yields almost nothing. Filter inside the engine, or over-fetch.
  • Skipping access control: one embedding space shared by tenants leaks data if the tenant filter is forgotten in even one code path. Enforce it in a single query wrapper.
  • Choosing a store before measuring: many applications with under a few hundred thousand vectors run fine on a flat scan or a Postgres extension.
  • Ignoring index build and memory cost: graph indexes can use more RAM than the raw vectors. Estimate before you provision.

Practice

Add a delete-by-filter method easy

Hint

Collect matching ids first, then delete, so you do not mutate the map while iterating.

Show solution

Loop over items, collect ids whose metadata matches every condition in where, then delete each id. Return the number removed. O(n) time. Collecting first avoids the classic bug of changing a collection during iteration.

Make query results diverse medium

Hint

After picking the best hit, penalise candidates very similar to hits already chosen.

Show solution

Over-fetch (say 20 candidates), then greedily select: take the best, and for each next pick the candidate maximising score - lambda * max_similarity_to_selected. This is maximal marginal relevance. It stops the top 5 from being five near-duplicate chunks of one page.

Design multi-tenant retrieval hard

Hint

Think about isolation, recall under filters, and the cost of one index per tenant.

Show solution

Options: one collection with a mandatory tenant_id filter, or one collection per tenant. A shared collection is cheaper and simpler to operate but relies on the filter being applied everywhere and can lose recall when a tenant owns a tiny fraction of the data. Per-tenant collections isolate data and give predictable recall, but cost more per small tenant and complicate operations. A common hybrid is a shared index for small tenants and dedicated indexes for the largest. Always enforce the tenant constraint in a single server-side query path, never in client code.

Interview questions

  1. What does a vector database provide beyond a numpy array? ANN indexing, metadata filtering, upserts and deletes, persistence, and scale features such as replication.
  2. How does metadata filtering interact with ANN search? Post-filtering can return too few results, pre-filtering may force slower exact scans, and filtered graph traversal over-fetches; selective filters can hurt recall or latency.
  3. pgvector versus a dedicated store? pgvector keeps vectors with relational data, transactions, and SQL, which is simpler to operate; a dedicated store can offer more tuning and scale at the cost of another system.
  4. What happens when you change the embedding model? All vectors must be re-embedded and the index rebuilt, which is why you keep source text and record the model version.

Interviewers look for: a clear boundary between the source of truth and the derived index, awareness of the filtering trade-offs, and a pragmatic "start simple, measure, then scale" stance.

Quick check

You run ANN to get the top 10, then drop results the user may not see. Users often get 2 results. What is the best fix?

Finished this lesson?

Further resources

Related lessons