Overall accuracy on LongMemEval — the industry-standard benchmark for long-context conversational memory. Single-session and multi-session recall, temporal reasoning, and knowledge-update questions, evaluated end-to-end against the published hypothesis format.
What LongMemEval actually tests
LongMemEval is the benchmark long-context memory systems get judged against — not a synthetic retrieval toy, but multi-session conversational histories with single-session questions, multi-session reasoning, temporal reasoning across sessions, and knowledge-update questions where an earlier fact is later corrected. A system that only does keyword lookup or flat vector similarity collapses on the multi-session and temporal categories. HIVEMIND's bi-temporal graph — memories that update, extend, derive from, and contradict each other with typed relationships — is built for exactly this shape of question.
LongMemEval question set + evaluation harness are public. Anyone can re-run the exact same questions against the exact same corpus.
The reported score is the full-set average, not a best-of-N run. Regressions below the pass/fail gate block deploys.
Every recall-path change re-runs the retrieval benchmark (P@5 / R@10 / NDCG@10 / MRR) before shipping — a regression fails the gate, not the customer.
The same Evaluation surface ships inside every HIVEMIND workspace — run the benchmark against your own memories, not just ours.
HIVEMIND vs. the alternatives
Keyword search fails the moment a question is phrased differently from the source text. Naive vector RAG improves recall but has no concept of time, contradiction, or multi-hop reasoning across sessions — it retrieves chunks, not a coherent memory. The gap widens exactly where enterprise memory matters most: knowing what changed, when, and why.
| Metric | HIVEMIND | Keyword search | Naive vector RAG |
|---|---|---|---|
| LongMemEval accuracy | 87.2% | 41.3% | 68.9% |
| Recall latency (p50) | <50ms | ~120ms | ~200ms |
| Multi-session reasoning | Native | None | Partial |
| Temporal / bi-temporal queries | Native | None | None |
| Contradiction resolution | Automatic | None | Manual |
Run it yourself
The published run lives at singulancelabs.com/benchmark. Every HIVEMIND workspace — personal, managed, or self-hosted — ships the same Evaluation surface: health overview, an interactive search tester with confidence and relevance scoring, and the retrieval benchmark itself. Don't take the number on faith — point it at your own memory and watch the gate run.