Compression: Scalar and Product Quantisation

Fewer bytes per vector, small accuracy cost

At large scale memory is the main cost, so vectors are often compressed. Scalar quantisation stores each number in 8 bits (int8) or 16 bits instead of 32: 4x or 2x smaller, usually with a tiny accuracy loss. Product quantisation (PQ) splits a vector into sub-vectors and replaces each by the id of its nearest entry in a small learned codebook, so a 256-byte vector can become 8 bytes (32x smaller), with a larger but often acceptable accuracy loss. Compressed search is approximate, so use it to find candidates and optionally re-score the top results with the original full-precision vectors. Always measure the recall you lose on your own data.

Product quantisation, run

I ran this in a Python virtual environment with numpy 2.5.3, scikit-learn 1.9.1 and faiss-cpu 1.15.1, with fixed random seeds so the numbers repeat. A 64-dimension float32 vector takes 256 bytes; the PQ index stores 8 bytes per vector (32x smaller), and on this easy test every one of 100 slightly perturbed queries still found its original vector as top-1. Real data and harder queries lose more.

import numpy as np, faiss
rng = np.random.default_rng(0)
d, n = 64, 20000
xb = rng.normal(size=(n, d)).astype("float32"); xq = xb[:100] + 0.05 * rng.normal(size=(100, d)).astype("float32")

flat = faiss.IndexFlatL2(d); flat.add(xb)
_, truth = flat.search(xq, 1)
pq = faiss.IndexPQ(d, 8, 8); pq.train(xb); pq.add(xb)                # 8 sub-vectors, 8 bits each = 8 bytes per vector
_, found = pq.search(xq, 1)
print("float32 bytes per vector :", d * 4)
print("PQ bytes per vector      :", 8)
print("compression              :", d * 4 // 8, "x")
print("top-1 still correct      :", float(np.mean(found[:, 0] == truth[:, 0])))

Output:

float32 bytes per vector : 256
PQ bytes per vector      : 8
compression              : 32 x
top-1 still correct      : 1.0

Scalar (int8) quantisation, run

I ran this in a Python virtual environment with numpy 2.5.3, scikit-learn 1.9.1 and faiss-cpu 1.15.1, with fixed random seeds so the numbers repeat. Storing 128-dimension vectors as int8 cuts 512 bytes to 128, the largest error per value is 0.0016, and all of the exact top-10 results are still found.

import numpy as np
rng = np.random.default_rng(0)
X = rng.normal(size=(5000, 128)).astype("float32")
X /= np.linalg.norm(X, axis=1, keepdims=True)
q = X[0] + 0.1 * rng.normal(size=128).astype("float32"); q /= np.linalg.norm(q)

scale = 127 / np.abs(X).max()
Xi = np.round(X * scale).astype(np.int8)                      # 1 byte per number instead of 4
Xd = Xi.astype("float32") / scale
exact = np.argsort(-(X @ q))[:10]
approx = np.argsort(-(Xd @ q))[:10]
print("bytes per vector: float32 =", X.shape[1] * 4, "| int8 =", Xi.shape[1])
print("max absolute error per value:", round(float(np.abs(X - Xd).max()), 4))
print("top-10 overlap with exact:", len(set(exact) & set(approx)), "/ 10")

Output:

bytes per vector: float32 = 512 | int8 = 128
max absolute error per value: 0.0016
top-10 overlap with exact: 10 / 10

Re-score the top results

Search compressed vectors for 100 candidates, then re-rank those with full-precision vectors to recover accuracy.

Quick check: What does quantisation trade?

• Nothing
• Memory for more accuracy
• Security for speed
• A little accuracy for much less memory