pgvector HNSW semantic search
Can we make sovereign memory retrieval fast enough to feel instant?
What it is
We are running pgvector with HNSW (Hierarchical Navigable Small World) indexing to power the semantic memory layer in MEOK. Each user's memories are stored as vector embeddings — the AI doesn't keyword-search; it finds memories by meaning. HNSW is the algorithm that makes this fast enough to be real-time.
What we're testing
Whether HNSW index parameters (ef_construction, m) can be tuned per-user without sacrificing retrieval quality. We're also testing how encrypted per-user vectors (AES-256 at rest) affect index build time at scale — this is a largely unsolved problem in production sovereign AI.
What we've learned
At 10,000 memories per user, HNSW retrieval at ef_search=40 returns in under 12ms on a standard Postgres instance. Cosine similarity at this scale is remarkably stable. The bigger challenge is index rebuild overhead when memories are deleted — we're working on incremental index updates.