Benchmark
Evaluated on KRB Benchmark v0.6.2 — open dataset, reproducible methodology, fixed baselines.
| System | Macro F1 | Tokens/query | Cost/1K queries |
|---|---|---|---|
| CKG | 0.471 | 269 | $7.81 |
| RAG (text-embedding-3-small) | 0.123 | 2,982 | $76.23 |
| GraphRAG (MS global mode) | 0.120 | — | — |
4× F1 · 11× fewer tokens · 5-hop F1 0.772 vs 0.170 · auditable by design
Why 5-hop reasoning matters
5-hop reasoning is where the gap compounds: retrieval degrades with each hop; graph traversal does not.
A dependency chain like LLM NIM → Triton → TensorRT-LLM → CUDA Toolkit → GPU Driver is a 5-hop question. RAG re-infers each link independently, accumulating error. The graph traverses the declared path — the answer is the same every time.
Why context efficiency collapses — and how CKG reverses it
Liu et al. formally quantify Context Transaction Cost (CTC): the compound tax paid every time context crosses an agent boundary. In multi-agent pipelines, CTC efficiency falls from 18.2 in Q1 to 1.6 by Q4 — 91% collapse with no model change.
CKG attacks all three root causes:
| CTC component | What it is | CKG's response |
|---|---|---|
| Token Latency Burden (τ) | Compute cost of transmitting context | 269 tokens instead of 2,982 |
| Handoff Cost (H) | Serialization loss at agent boundaries | get_prerequisites() replaces re-retrieval |
| Compression Loss (C) | Information destroyed when context is summarized | The graph is the compressed form — done once, offline |
Structured context doesn't consume your context window. It opens it.
Reproduce it
Benchmark dataset: huggingface.co/datasets/danyarm/ckg-benchmark
Paper: github.com/Yarmoluk/ckg-benchmark/blob/main/paper/main.pdf