Vector Databases
What a vector database adds beyond an array of embeddings: indexing, metadata filtering, upserts, and persistence. Build a tiny store and compare pgvector, dedicated stores, and FAISS.
intermediate40 min#vector-database#pgvector#faiss#metadata-filtering#upsert
Prerequisites: Vectors and Embeddings, Similarity Search and ANN Indexes
Why it matters
You can get surprisingly far with a Python list of vectors and a top_k function. Then real life arrives: documents are edited, so vectors must be replaced. Users may only see their own company's documents, so results must be filtered. The process restarts and the vectors must survive. There are fifty million of them and brute force is too slow. A vector database is the layer that handles these problems so your application code does not have to.
Intuition
Think of a library. An array of embeddings is a pile of books on the floor with a note saying which are about cooking. A vector database is the library: shelves (an index) so you do not scan every book, a catalogue with author and year (metadata) so you can ask for "cooking, after 2020", a librarian who swaps an old edition for a new one (upserts), and a building that is still there tomorrow (durability).
Crucially, the library does not change what a good book is. Similarity still comes from embeddings. The database only makes finding and managing them practical.
How it works
What a vector store adds over a plain array:
- An ANN index. HNSW or IVF as described in the previous lesson, so queries avoid scanning everything. Most stores let you choose the index and tune its knobs.
- Metadata and filtering. Each vector carries a payload such as
tenant,lang,created_at, orsource. Queries combine similarity with conditions on that payload. - Upserts and deletes. Writing an id that exists replaces it. This matters because a re-embedded document must not leave a stale copy behind.
- Persistence. Data lives on disk with a write-ahead log or snapshots, and can be restored after a crash.
- Scale and operations. Replication, sharding, backups, access control.
Filtering is the subtle part. There are three strategies:
- Post-filter: run ANN for the top
k, then drop non-matching results. Fast, but with a selective filter you may get fewer thankresults, or none. - Pre-filter: find matching ids first, then search only those. Correct, but a graph index built over everything cannot easily skip nodes, so engines either fall back to an exact scan when the filter is very selective or navigate the graph while checking the filter as they go.
- Filtered traversal: the search itself skips non-matching candidates, and many systems over-fetch candidates (
eflarger thank) to compensate.
Good systems pick between these automatically, but you should still know that heavy filters can lower recall or raise latency.
Where to keep vectors: principles, not products.
- A library such as FAISS gives you fast index structures in memory and nothing else. You manage ids, metadata, persistence, updates, and concurrency. Great for research, batch jobs, and embedded use; a lot of plumbing for a multi-user service.
- An extension to a database you already run (for example pgvector for Postgres) keeps vectors next to your relational rows. You get transactions, joins, backups, and SQL filters for free, and one system to operate. The trade-off is that very large collections or very high query rates may stress a general-purpose database more than a purpose-built engine.
- A dedicated vector database is built around ANN: rich filtering, sharding, tuning, and often managed hosting. The trade-off is another system to run, keep in sync with your source of truth, and pay for.
A reasonable default: start with brute force or your existing database. Move to a dedicated engine when scale, latency, or filtering needs prove it, not before. Whichever you pick, keep the source of truth (the original text and ids) in your primary database, so the vector store can always be rebuilt, for example after changing embedding models.
| Operation | Time | Notes |
|---|---|---|
| Insert / upsert one vector | O(d) flat, ~O(log n * d) graph index | Graph insert must link neighbours |
| Delete by id | O(1) flat; tombstone in graph | Graphs usually mark deleted, rebuild later |
| Query, no filter | O(n * d) flat, ~O(log n * d) ANN | Approximate for ANN |
| Query with metadata filter | Depends on selectivity | Pre/post/filtered traversal |
| Snapshot to disk | O(n * d) | Write every vector once |
Code: a tiny in-memory vector store
We need vectors, and a real embedding model would be a network call. Instead, embed below is a small stand-in: it hashes each word into one of 32 buckets and normalises the result. It captures word overlap, not meaning, but it is deterministic and good enough to exercise the store. To use a real model, replace only the body of embed with a call to your embedding provider; everything else stays the same.
import json
import math
import re
DIM = 32
def embed(text):
"""Stand-in for a real embedding model: hashed bag of words, unit length."""
vec = [0.0] * DIM
for word in re.findall(r"[a-z0-9]+", text.lower()):
h = 2166136261
for ch in word:
h = ((h ^ ord(ch)) * 16777619) % 4294967296 # FNV-1a
vec[h % DIM] += 1.0
n = math.sqrt(sum(x * x for x in vec)) or 1.0
return [x / n for x in vec]
class VectorStore:
def __init__(self):
self.items = {} # id -> {"vec": [...], "text": str, "meta": dict}
def upsert(self, id, text, meta=None):
self.items[id] = {"vec": embed(text), "text": text, "meta": meta or {}}
def delete(self, id):
return self.items.pop(id, None) is not None
def query(self, text, k=3, where=None):
q = embed(text)
scored = []
for id, item in self.items.items():
if where and any(item["meta"].get(f) != v for f, v in where.items()):
continue # pre-filter: skip non-matching items
score = sum(a * b for a, b in zip(q, item["vec"]))
scored.append((-score, id))
scored.sort()
return [(id, -neg) for neg, id in scored[:k]]
def dumps(self):
return json.dumps(self.items)
@staticmethod
def loads(blob):
store = VectorStore()
store.items = json.loads(blob)
return store
def show(label, results):
print(label, " ".join(f"{id}:{score:.3f}" for id, score in results))
store = VectorStore()
store.upsert("d1", "How to reset your password", {"team": "support"})
store.upsert("d2", "Forgotten password reset by email", {"team": "support"})
store.upsert("d3", "Quarterly revenue report for finance", {"team": "finance"})
store.upsert("d4", "Pasta recipe with tomato sauce", {"team": "food"})
store.upsert("d5", "Password policy for employees", {"team": "security"})
q = "forgot my password reset"
show("all ", store.query(q))
show("support ", store.query(q, where={"team": "support"}))
store.upsert("d2", "Quarterly password rotation schedule", {"team": "security"})
show("upserted", store.query(q))
store.delete("d1")
show("deleted ", store.query(q))
restored = VectorStore.loads(store.dumps())
print("restored matches:", restored.query(q) == store.query(q))const DIM = 32;
// Stand-in for a real embedding model: hashed bag of words, unit length.
function embed(text) {
const vec = new Array(DIM).fill(0);
for (const word of text.toLowerCase().match(/[a-z0-9]+/g) || []) {
let h = 2166136261;
for (const ch of word) {
h = Math.imul(h ^ ch.charCodeAt(0), 16777619) >>> 0; // FNV-1a
}
vec[h % DIM] += 1;
}
const n = Math.sqrt(vec.reduce((s, x) => s + x * x, 0)) || 1;
return vec.map((x) => x / n);
}
class VectorStore {
constructor() {
this.items = new Map(); // id -> { vec, text, meta }
}
upsert(id, text, meta = {}) {
this.items.set(id, { vec: embed(text), text, meta });
}
delete(id) {
return this.items.delete(id);
}
query(text, k = 3, where = null) {
const q = embed(text);
const scored = [];
for (const [id, item] of this.items) {
if (where && Object.entries(where).some(([f, v]) => item.meta[f] !== v)) {
continue; // pre-filter: skip non-matching items
}
const score = q.reduce((s, a, i) => s + a * item.vec[i], 0);
scored.push([-score, id]);
}
scored.sort((a, b) => a[0] - b[0] || (a[1] < b[1] ? -1 : a[1] > b[1] ? 1 : 0));
return scored.slice(0, k).map(([neg, id]) => [id, -neg]);
}
dumps() {
return JSON.stringify([...this.items]);
}
static loads(blob) {
const store = new VectorStore();
store.items = new Map(JSON.parse(blob));
return store;
}
}
function show(label, results) {
console.log(label, results.map(([id, s]) => `${id}:${s.toFixed(3)}`).join(" "));
}
const store = new VectorStore();
store.upsert("d1", "How to reset your password", { team: "support" });
store.upsert("d2", "Forgotten password reset by email", { team: "support" });
store.upsert("d3", "Quarterly revenue report for finance", { team: "finance" });
store.upsert("d4", "Pasta recipe with tomato sauce", { team: "food" });
store.upsert("d5", "Password policy for employees", { team: "security" });
const q = "forgot my password reset";
show("all ", store.query(q));
show("support ", store.query(q, 3, { team: "support" }));
store.upsert("d2", "Quarterly password rotation schedule", { team: "security" });
show("upserted", store.query(q));
store.delete("d1");
show("deleted ", store.query(q));
const restored = VectorStore.loads(store.dumps());
console.log(
"restored matches:",
JSON.stringify(restored.query(q)) === JSON.stringify(store.query(q)) ? "True" : "False"
);Both versions print:
all d1:0.447 d2:0.447 d5:0.250
support d1:0.447 d2:0.447
upserted d1:0.447 d2:0.408 d5:0.250
deleted d2:0.408 d5:0.250 d3:0.224
restored matches: True
Read the output as a story: the unfiltered query finds the password documents; the filter keeps only the support team; the upsert replaced d2 (same id, new text and metadata), so its score dropped and it left the support team; the delete removed d1; and a snapshot restored through JSON gives identical answers. The last line of the deleted result shows a weak hit, d3, only because two unrelated words happened to hash to the same bucket, a reminder that this stand-in is crude. Note that this store does an exact scan, so it is flat and correct but O(n * d) per query. A production engine would put an ANN index inside query.
Common pitfalls
- Treating the vector store as the source of truth: keep original documents elsewhere so you can rebuild after a model change or a corrupted index.
- Forgetting upsert semantics: adding with a fresh id on every edit leaves old versions that still match queries. Use stable ids such as
doc_id#chunk_no. - Over-filtering with post-filter search: asking for 5 results and then discarding 4 of them for permissions yields almost nothing. Filter inside the engine, or over-fetch.
- Skipping access control: one embedding space shared by tenants leaks data if the tenant filter is forgotten in even one code path. Enforce it in a single query wrapper.
- Choosing a store before measuring: many applications with under a few hundred thousand vectors run fine on a flat scan or a Postgres extension.
- Ignoring index build and memory cost: graph indexes can use more RAM than the raw vectors. Estimate before you provision.
Practice
Add a delete-by-filter method easy
Hint
Collect matching ids first, then delete, so you do not mutate the map while iterating.
Show solution
Loop over items, collect ids whose metadata matches every condition in where, then delete each id. Return the number removed. O(n) time. Collecting first avoids the classic bug of changing a collection during iteration.
Make query results diverse medium
Hint
After picking the best hit, penalise candidates very similar to hits already chosen.
Show solution
Over-fetch (say 20 candidates), then greedily select: take the best, and for each next pick the candidate maximising score - lambda * max_similarity_to_selected. This is maximal marginal relevance. It stops the top 5 from being five near-duplicate chunks of one page.
Design multi-tenant retrieval hard
Hint
Think about isolation, recall under filters, and the cost of one index per tenant.
Show solution
Options: one collection with a mandatory tenant_id filter, or one collection per tenant. A shared collection is cheaper and simpler to operate but relies on the filter being applied everywhere and can lose recall when a tenant owns a tiny fraction of the data. Per-tenant collections isolate data and give predictable recall, but cost more per small tenant and complicate operations. A common hybrid is a shared index for small tenants and dedicated indexes for the largest. Always enforce the tenant constraint in a single server-side query path, never in client code.
Interview questions
- What does a vector database provide beyond a numpy array? ANN indexing, metadata filtering, upserts and deletes, persistence, and scale features such as replication.
- How does metadata filtering interact with ANN search? Post-filtering can return too few results, pre-filtering may force slower exact scans, and filtered graph traversal over-fetches; selective filters can hurt recall or latency.
- pgvector versus a dedicated store? pgvector keeps vectors with relational data, transactions, and SQL, which is simpler to operate; a dedicated store can offer more tuning and scale at the cost of another system.
- What happens when you change the embedding model? All vectors must be re-embedded and the index rebuilt, which is why you keep source text and record the model version.
Interviewers look for: a clear boundary between the source of truth and the derived index, awareness of the filtering trade-offs, and a pragmatic "start simple, measure, then scale" stance.
Quick check
You run ANN to get the top 10, then drop results the user may not see. Users often get 2 results. What is the best fix?
Finished this lesson?