External KV Array vs Local NVMe vs NFS: A Three-Way Measured Comparison
Same-workload measurements: FX100 beats a local NVMe drive by 26–32% on TTFT and 29–40% on throughput; beats no-external-storage re-compute by 8.6–20×; beats NFS model loading by 6.2–9.3×. All three baselines are report-verifiable.
Three-way data (480B, TP8, cold recovery)
Concurrency 8: FX100 TTFT p50 7.53 s / 56.6 tok/s; local drive 10.17 s / 43.9 tok/s. Concurrency 16: FX100 11.85 s / 74.9; local drive 17.31 s / 53.6; no-external-storage re-compute 149.48 s / 4.1. Concurrency 32: FX100 26.35 s / 71.6; local drive 35.73 s / 53.9 (measured, R2; re-compute measured at conc 16 only).
The NFS control is on the Ascend platform (R9): DeepSeek-32B/70B serving loads gap 6.2× / 9.3×, with training checkpoints and data reads similarly improved.
When a local drive or NFS is actually the right call
Honestly: single-node small models, low concurrency, no session-recovery needs — a local NVMe drive is entirely sufficient; no array needed. NFS suits cold, infrequently accessed data and config distribution. The array's domain is production clusters with multi-node sharing, long-context high concurrency, and frequent model loading/switching — precisely what the joint-test gates verify (stop-loss if unmet).
FAQ
Wouldn't multi-drive local RAID0 catch up with the array?
Bandwidth can come close, but caches cannot be shared across nodes, capacity is not pooled, and every node duplicates the build. The cross-instance fs:// hot sharing measured in R3 is a structural gap for local schemes.
Can these comparisons be reproduced?
Yes: the R8 code export contains load clients, orchestration scripts and raw data; platform and software versions are published in full (8× MI308X, ROCm 7.2, vLLM 0.20.1).
Why was the re-compute baseline measured only at concurrency 16?
The report records it honestly: at that level re-compute already hit TTFT 149.5 s and 4.1 tok/s — other concurrency levels carried no practical reference value, and the report notes this basis.
Data sources (verifiable)
Related reading
- NVMe-oF Inference Storage: Measured Results of a RoCEv2 All-Flash Array
- TTFT Optimization: Engineering Paths to Cut LLM First-Token Latency
- Model-Load Acceleration: A Measured Path from 23 Minutes to 2.5 Minutes
Joint test first, decisions second: gate-based acceptance with built-in stop-loss
The full costing model is provided as reproducible Python after NDA — customers can rerun it with their own parameters. Every key figure on this site carries a report ID and is open to third-party verification.
This site presents business-cooperation information and constitutes neither an investment offer nor any promise of returns. Measured data come from signed / official test reports (see the Evidence Library); vendor specs, public sources and estimates are labeled as such.