Prefill/Decode Disaggregation and the KV Cache Transport Layer
PD disaggregation puts prefill and decode on different instances, making the KV cache the hand-off object between them. Disaggregation amplifies the KV tier's importance: inter-instance transport plus external persistence for cold sessions.
Why PD disaggregation is hot, and what it has to do with KV
Prefill is compute-bound; decode is memory-bandwidth-bound; mixing them on one instance causes interference. The mainstream industry direction (NVIDIA Dynamo, Mooncake and other public architectures) separates the two — at the price that prefill-produced KV must be shipped to decode instances. KV cache graduates from "instance-internal state" to first-class cluster data.
Once KV flows across instances, a shared storage tier becomes pivotal: R3 verified fs:// shared-pool cross-instance hot sharing — any instance can hit the same KV copy, exactly the substrate a disaggregated architecture needs.
Where the external tier sits in a disaggregated design
The hot path (prefill → decode immediate hand-off) rides RDMA memory transfer; the warm/cold path (session suspension, recovery, cross-batch reuse) lands on external all-flash. FX100 provides the latter over NVMe-oF/RoCEv2: measured 480B production-form cold-recovery read-back TTFT p50 11.85 s, 12.6× vs re-compute (R2).
The ratio anchor applies here too: a single 100 GbE port sustains 8-node KV read-back (R2/R3 bandwidth data). Under disaggregation, prefill and decode pools can share one external tier for higher capacity utilization.
FAQ
Is PD disaggregation always better?
No: at small scale it adds complexity and transport overhead. Its sweet spot is large multi-tenant serving; single machines and small clusters get more direct gains from KV tiering first (R2 single-instance basis: +29–40%).
Does Mingxin support frameworks like Dynamo / Mooncake?
FX is a standard NVMe-oF block device — any framework's SSD/file sink layer can attach. The vLLM+LMCache path is already measured (R1–R3); other frameworks are validated on joint-test schedules, with no numbers promised in advance.
Will KV transport become the new bottleneck?
It depends on network and parallelism design. Measured anchors: 5.23 GB/s per-GPU cold read after the parallel-read patch (R1), and 8-node shared read-back approaching the NIC ceiling (G3 gate basis).
Data sources (verifiable)
Related reading
- Mingxin FX and NVIDIA Dynamo: Complementary, Not Competing
- KV Cache Tiering: The Three-Level HBM → RAM → All-Flash Architecture
- NVMe-oF Inference Storage: Measured Results of a RoCEv2 All-Flash Array
Joint test first, decisions second: gate-based acceptance with built-in stop-loss
The full costing model is provided as reproducible Python after NDA — customers can rerun it with their own parameters. Every key figure on this site carries a report ID and is open to third-party verification.
This site presents business-cooperation information and constitutes neither an investment offer nor any promise of returns. Measured data come from signed / official test reports (see the Evidence Library); vendor specs, public sources and estimates are labeled as such.