# Comparing Engines: A Practical Checklist — Vector Databases

Source: https://www.geekswithgeeks.com/en/vector-databases/e-compare

> Ask the same questions of every candidate.

## Questions beat feature tables

Feature lists on vendor sites are similar and change monthly, so compare with **questions about your workload**. **Scale**: how many vectors and what dimension now and in two years; does it fit in memory; what does it do beyond RAM (disk indexes, quantisation)? **Filtering**: are filters evaluated during search or after; are payload indexes supported; how does it behave with a 1% filter? **Search types**: dense only, or sparse/keyword and hybrid with fusion; multi-vector; reranking hooks? **Updates**: how quickly does a write become searchable; how are deletes and re-indexing handled? **Consistency and durability**: transactions, write-ahead log, snapshots, replication? **Operations**: deployment (Kubernetes, managed), backups, monitoring, upgrades. **Security**: authentication, role-based access, encryption, tenant isolation, audit. **Ecosystem and cost**: client libraries, framework integrations, licence, support, total price. Score each candidate against **your** answers.

## A scoring sheet

Fill it in with measured results from your own tests, not from marketing pages.

```text
Criterion (weight)            Candidate A     Candidate B     Candidate C   how measured
recall@10 at p95 < 50 ms (5)   __/5            __/5            __/5          your golden set + load test
filter at 1% selectivity (4)   __/5            __/5            __/5          returns 10 rows? latency?
write-to-searchable delay (3)  __/5            __/5            __/5          insert, poll until found
ops effort (3)                 __/5            __/5            __/5          deploy, backup, upgrade drill
security features (4)          __/5            __/5            __/5          RBAC, encryption, tenant tests
cost at 2-year scale (4)       __/5            __/5            __/5          price sheet + memory estimate
```

**Quiz:** Why compare engines with questions about your workload rather than feature tables?

- [x] Feature lists look alike and change often; behaviour on your data decides
- [ ] Features never matter
- [ ] Vendors forbid comparison
- [ ] Tables are always wrong

*Answer:* Feature lists look alike and change often; behaviour on your data decides. Test with your vectors, filters and load before choosing.