Case Analysis of Huawei OceanStor UCM Inference Acceleration Solution

Huawei OceanStor A-series UCM (Unified Cache Management) is a storage acceleration solution designed for large-model inference scenarios. Its core approach is to offload KV Cache and model weights from GPU memory to a high-performance storage pool, alleviating memory bottlenecks through a disaggregated storage-compute architecture. This article analyzes the technical characteristics and selection considerations of the UCM solution, referencing measured data from Mingxin FX100 under similar workloads, for the reference of computing center technical decision-makers.

Background: Why Inference Acceleration Needs Storage-Side Solutions

The throughput and latency bottlenecks of large-model inference are largely determined not by compute power but by memory capacity and bandwidth. According to the analysis in "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness" (arXiv:2205.14135), attention computation is limited by HBM bandwidth rather than compute, making IO-aware optimization a key path to efficiency. When model size exceeds single-GPU memory, the storage and retrieval of KV Cache becomes the core contradiction in system design.

Huawei OceanStor A-series UCM addresses this by tiering KV Cache: hot data remains in GPU memory, warm data is offloaded to the storage array, and cold data is persisted to disk. This aligns with the KVCache-centric disaggregated architecture proposed in "Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving" (arXiv:2407.00079), and is consistent with the rationale for KV Cache paging management described in "Efficient Memory Management for Large Language Model Serving with PagedAttention" (arXiv:2309.06180)—memory fragmentation and capacity limits are practical problems that must be solved.

Technical Path and Applicability Conditions of the UCM Solution

UCM relies on the GPUDirect Storage mechanism. According to NVIDIA GPUDirect Storage Documentation, this technology allows GPUs to access storage devices directly, bypassing CPU memory and reducing data copy overhead. Huawei OceanStor A-series connects to GPU servers via NVMe-oF (RoCEv2) networks, keeping KV Cache read latency within the range that storage media can provide.

This path has three applicability conditions: First, the random read bandwidth of the storage array must be sufficiently high; otherwise, offloaded KV Cache reads become a new bottleneck. Second, network latency must be at microsecond levels; RoCEv2 significantly reduces transmission overhead compared to TCP/IP. Third, the framework side must support automatic tiered scheduling between memory and storage, which depends on coordination with inference frameworks such as vLLM and LMCache.

Measured Comparison: UCM-Type Solutions vs. Mingxin FX100 Quantitative Data

To evaluate the practical benefits of UCM-type solutions, the following references measured data from Mingxin FX100 under the same workload type (480B MoE model, long-context cold recovery). Note: Huawei has not published independent benchmark numbers for UCM; the comparison below uses Mingxin's measured data as the reference, with UCM discussed only qualitatively.

Under a 480B·TP8 configuration, Mingxin FX100 achieves 29–40% throughput improvement and 26–32% TTFT reduction from KV tiered acceleration (measured, reports R2/R3). On the Huawei Atlas 910B platform, FX100 delivers 6.2–9.3× model loading acceleration over the NFS baseline (measured, report R9), providing a reference order of magnitude for storage acceleration in the Ascend ecosystem.

Selection Considerations: Trade-offs Between UCM and Independent Storage Acceleration Solutions

For computing center decision-makers, UCM and independent storage acceleration solutions like Mingxin FX100 are not mutually exclusive but can be layered:

First, deployment complexity. UCM is deeply tied to Huawei OceanStor A-series hardware and software stacks, making it suitable for customers already using Huawei's full-stack solutions. FX100, as a standard PCIe/NVMe-oF device, can be integrated into existing x86 or ARM servers and directly interfaces with open-source frameworks like vLLM and LMCache, offering a relatively lower deployment barrier.

Second, performance verification. Huawei UCM has not published reproducible benchmark figures. While MLPerf Inference (published by MLCommons) provides public inference performance benchmarks, it does not separately cover KV Cache offload scenarios. Mingxin FX100's measured data comes from signed test reports (R1–R9), and it offers approximately 10 weeks of gated joint testing (from G1 arrival acceptance to G4 72-hour stability), with stop-loss provisions if targets are not met—providing a verifiable evaluation path for technology selection.

Third, ecosystem compatibility. UCM optimizations require deep coordination with the Huawei Ascend platform. FX100 has measured data on both AMD MI308X (ROCm 7.2) and Huawei 910B platforms, covering a broader hardware ecosystem. According to PyTorch Documentation, framework-side memory management behavior directly affects the effective scope of offload strategies; during selection, confirm the compatibility declaration between the target framework version and the storage solution.

Conclusion

Huawei OceanStor UCM represents a typical path for storage vendors entering inference acceleration—alleviating memory capacity bottlenecks through storage-compute disaggregation and KV Cache tiering. Its applicability depends on storage bandwidth, network latency, and framework coordination all meeting requirements simultaneously. Mingxin FX100 provides reproducible quantitative benefits under similar workloads (throughput +29–40%, TTFT ↓26–32%, measured in reports R2/R3) and supports gated joint testing, making it suitable as a comparison or alternative to the UCM solution. Decision-makers are advised to use neutral benchmarks such as MLPerf as the reference standard, combined with their own workload characteristics and existing hardware stack, and complete quantifiable comparative testing before actual deployment.

Key Q&A

Q: What is the core technical path of the Huawei OceanStor UCM solution? A: UCM uses a disaggregated storage-compute architecture to tier and offload KV Cache to storage arrays, relying on GPUDirect Storage to reduce data path overhead. Its design philosophy aligns with public research such as Mooncake and PagedAttention.

Q: What are the measured benefits of Mingxin FX100 under similar workloads? A: Under 480B model long-context cold recovery workloads, KV tiered acceleration improves inference throughput by 29–40% and reduces TTFT by 26–32% (measured, reports R2/R3); it achieves 8.6–20× speedup over the no external memory recompute baseline.

Q: How should UCM and FX100 be selected? A: UCM suits Huawei full-stack customers; FX100 suits scenarios requiring cross-platform compatibility and verifiable performance. FX100 offers gated joint testing (with in-band verification of metrics such as TTFT reduction ≥25%), with stop-loss provisions if targets are not met.

References

1. MLPerf Inference: Datacenter Benchmark Suite Results — https://mlcommons.org/benchmarks/inference-datacenter/
2. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — https://arxiv.org/abs/2205.14135
3. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — https://arxiv.org/abs/2407.00079
4. Efficient Memory Management for Large Language Model Serving with PagedAttention — https://arxiv.org/abs/2309.06180
5. NVIDIA GPUDirect Storage Documentation — https://docs.nvidia.com/gpudirect-storage/index.html
6. PyTorch Documentation — https://pytorch.org/docs/stable/index.html