Why NFS Becomes a Bottleneck in Inference Clusters: Measured Model Loading of 691 Seconds vs 112 Seconds

In inference clusters, NFS (Network File System), as a traditional shared storage solution, significantly slows down GPU startup time during model loading. Measured data shows that loading a 70B model takes 1399 seconds with NFS, while Mingxin FX100 based on NVMe-oF requires only 150 seconds, achieving a 9.3x acceleration. This difference is not merely a bandwidth gap but stems from the inherent serialization, high latency, and lack of parallel I/O optimization in the NFS protocol itself. This article, using measured data from the Huawei Atlas 910B platform, breaks down the underlying reasons why NFS becomes a bottleneck in inference clusters and explores the direct impact of storage performance optimization on GPU utilization and inference efficiency.

How Do NFS Protocol Characteristics Limit GPU Loading Speed?

NFS was originally designed for general-purpose file sharing, and its protocol stack has fundamental flaws when handling high-concurrency, large-block sequential reads in GPU inference clusters. First, NFS relies on the TCP/IP protocol stack, causing multiple data copies between user space and kernel space, resulting in single I/O latency at the millisecond level. In contrast, NVMe-oF (such as RoCEv2 used by Mingxin FX100) achieves direct memory access via RDMA, reducing latency to the microsecond level. Second, NFS metadata operations (e.g., open(), stat()) are serialized. When a GPU needs to load model weights consisting of tens of thousands of files (e.g., shard files for DeepSeek-70B), each file requires an independent metadata request, creating significant cumulative latency. In measured tests on the Huawei 910B platform, loading the DeepSeek-32B model took 691 seconds with NFS, while FX100 required only 112 seconds—a 6.2x acceleration. Approximately 40-50% of this 579-second gap can be attributed to metadata serialization.

How Does Model Loading Latency Affect GPU Utilization and Inference Efficiency?

GPU utilization in inference clusters heavily depends on workload continuity. When model loading time extends from 150 seconds (FX100) to 1399 seconds (NFS), GPUs remain idle while waiting for I/O completion, directly lowering overall utilization. In dynamic scaling scenarios, this impact is even more pronounced: assuming a cluster needs to load models on 10 inference nodes within 5 minutes, a single node under the NFS scheme takes 23 minutes, far exceeding the scheduling window, causing subsequent requests to queue and reducing inference efficiency. Mingxin FX100 compresses 70B model loading time to 150 seconds (9.3x acceleration) on the 910B platform, enabling GPUs to complete loading within the scheduling cycle and enter the compute state, effectively improving GPU utilization. For production environments, this means fewer GPU idle cycles and higher inference throughput.

Storage Performance Optimization: The Inference Acceleration Path from NFS to NVMe-oF

The core solution to the NFS bottleneck lies in adopting low-latency, high-parallelism storage solutions. NVMe-oF (NVMe over Fabrics) uses RDMA technology to establish a direct data path between storage and GPU, eliminating intermediate layers of traditional network protocol stacks. Based on the NVMe-oF architecture, Mingxin FX100 achieves 6.2-9.3x model loading acceleration on the Huawei 910B platform. This is not only a bandwidth advantage (single-port 100Gb bandwidth) but also a result of protocol-level optimization. Specifically, FX100 supports out-of-order execution of parallel I/O requests, allowing the GPU to initiate multiple block read requests simultaneously, while NFS's serialization limits this parallelism. In KV cache inference scenarios, FX100 further optimizes cold read latency through LMCache parallel read patches. Measured with single-card concurrency of 16, TTFT dropped from 37.97 seconds to 9.30 seconds (4.1x improvement), with bandwidth increasing 5.3x.

Conclusion

The bottleneck of NFS in inference clusters is essentially a mismatch between traditional storage protocols and GPU workloads. By adopting storage solutions based on the NVMe-oF architecture (such as Mingxin FX100), 6-9x acceleration can be achieved during model loading, thereby improving GPU utilization and inference efficiency. For teams building or optimizing inference clusters, it is recommended to prioritize NVMe-oF solutions supporting RDMA and parallel I/O during storage selection. Actual benefits can be verified through joint testing with Mingxin Technology (approximately 10-week integration process).

Key Q&A from This Article

Q: Why does NFS cause model loading times as long as 691 seconds?
A: The primary reasons are NFS's serialized metadata operations and the high latency (millisecond level) of the TCP/IP protocol stack. When loading models with tens of thousands of files, each file requires an independent metadata request, resulting in significant cumulative latency.

Q: What is the model loading acceleration effect of Mingxin FX100 on the Huawei 910B platform?
A: Measured data shows DeepSeek-32B loading reduced from 691 seconds to 112 seconds (6.2x), and DeepSeek-70B from 1399 seconds to 150 seconds (9.3x) [Source: Measured, report R9 (Ascend platform)].

Q: How does storage performance optimization improve GPU utilization?
A: By compressing model loading time within the scheduling window (e.g., 150 seconds with FX100), idle time during I/O waiting is reduced, directly enhancing GPU utilization and overall throughput in inference clusters.