Model-Load Acceleration: A Measured Path from 23 Minutes to 2.5 Minutes
Model loading is bounded by storage read bandwidth: over NFS, loading a 70B model takes 23 minutes. Mingxin FX100 measured DeepSeek-70B loading down to 150 seconds (9.3×) and 32B down to 112 seconds (6.2×).
The cost of slow loading
Service restarts, elastic scale-out, model hot updates, failure migration — each rereads hundreds of GB of weights. NFS shared storage is the common answer, but protocol overhead and single-stream bandwidth limits stretch load times to tens of minutes while GPUs idle throughout.
R9, measured on Huawei Atlas 910B ×8: DeepSeek-32B serving load 691 s → 112 s (6.2×); DeepSeek-70B 1399 s → 150 s (9.3×) — the control group being exactly the NFS baseline.
Concurrent loading and model switching
Multi-GPU concurrent loading is another trap: 8 GPUs cold-reading the same weights congest the storage side. R1 measured FX100 delivering 16% faster wall-clock for 8-GPU concurrent loads and 30% faster single-model loads.
For multi-tenant platforms that switch models frequently, switching idle time eats compute directly: R1's measured modeling shows effective utilization rising from 46.7% to 62.8% at the 20-switches/hour level.
FAQ
Does this hold on NVIDIA platforms too?
The bottleneck mechanism (storage bandwidth vs idle GPUs) is platform-independent; this data set was measured on Ascend 910B (R9) and AMD MI308X (R1), labeled by platform and never mixed.
What about caching weights on local drives?
Local drives need a copy per node and full redistribution on updates; an external pool shares one copy cluster-wide with instant updates and lower capacity cost.
Any benefit for training?
Yes: training checkpoint concurrent writes measured 1.9× faster (178 s → 94 s, 8-GPU 32B LoRA full-model snapshots, R1); training-data read acceleration is in R9 (YOLOv8/COCO group).
Data sources (verifiable)
Related reading
- AI Inference Storage Acceleration: The Third Way to Raise Cluster Output Without Adding GPUs
- NVMe-oF Inference Storage: Measured Results of a RoCEv2 All-Flash Array
- KV Cache Tiering: The Three-Level HBM → RAM → All-Flash Architecture
Joint test first, decisions second: gate-based acceptance with built-in stop-loss
The full costing model is provided as reproducible Python after NDA — customers can rerun it with their own parameters. Every key figure on this site carries a report ID and is open to third-party verification.
This site presents business-cooperation information and constitutes neither an investment offer nor any promise of returns. Measured data come from signed / official test reports (see the Evidence Library); vendor specs, public sources and estimates are labeled as such.