Capacity Planning: Memory Is the Budget

Vectors + graph + payload, times copies

Estimate in four parts: vector data (vectors × dimensions × bytes per number), index overhead (for HNSW roughly 2 × M × 4 bytes of links per vector, plus bookkeeping), payload/metadata (often significant if you store text), and replicas (multiply by the copy count). Then add headroom (30 to 50%) for growth, merges, and the OS cache. If it does not fit: quantise (int8, product quantisation) and keep full-precision vectors on disk for re-scoring; use fewer dimensions or a smaller model; keep payload text in another store; shard; or use a disk-based index. This is a rough planning model: real engines add their own overhead, so confirm with a load test on a representative sample before you buy hardware.

A capacity estimate, run

I ran this plain-Python (standard library only) example. The formula adds vectors, an estimated HNSW link overhead (M=16) and 500 bytes of payload per record, then doubles for 2 replicas. Ten million 768-dimension float32 vectors need about 37 GB for one copy and 74 GB with replicas; storing the numbers as int8 cuts one copy to 14 GB. These are planning estimates, not measurements of a specific engine.

def gb(x): return x / 1e9

def plan(n_vectors, dims, bytes_per=4, hnsw_m=16, replicas=2, payload_bytes=500):
    vectors = n_vectors * dims * bytes_per
    graph = n_vectors * hnsw_m * 2 * 4              # rough: ~2*M neighbour ids (4 bytes each) per vector
    payload = n_vectors * payload_bytes
    one_copy = vectors + graph + payload
    return one_copy, one_copy * replicas

for n, d, b in [(1_000_000, 768, 4), (10_000_000, 768, 4), (10_000_000, 768, 1), (100_000_000, 384, 1)]:
    one, total = plan(n, d, b)
    print(f"{n:>11,} x {d} dims, {b} byte/number: one copy {gb(one):7.1f} GB | with 2 replicas {gb(total):7.1f} GB")

Output:

  1,000,000 x 768 dims, 4 byte/number: one copy     3.7 GB | with 2 replicas     7.4 GB
 10,000,000 x 768 dims, 4 byte/number: one copy    37.0 GB | with 2 replicas    74.0 GB
 10,000,000 x 768 dims, 1 byte/number: one copy    14.0 GB | with 2 replicas    27.9 GB
100,000,000 x 384 dims, 1 byte/number: one copy   101.2 GB | with 2 replicas   202.4 GB

Quick check: What should you do after a paper capacity estimate?

• Confirm with a load test on a representative sample
• Buy hardware immediately
• Ignore overhead
• Skip replicas