Ascend 910B vs. NVIDIA GPU: Inference Storage Energy Consumption Comparison
The energy consumption difference between the Ascend 910B platform and NVIDIA GPUs in inference storage systems stems not from chip compute power
The energy consumption difference between the Ascend 910B platform and NVIDIA GPUs in inference storage systems stems not from chip compute power, but from storage data paths and memory access patterns. Multiple measurements conducted by Mingxin on its own test platforms show that storage-side energy consumption accounts for a significant share in inference scenarios, and the impact of storage access efficiency optimization on end-to-end energy consumption often outweighs differences in GPU model specifications. This article compares the two platforms across three dimensions: storage paths, KV Cache memory access, and loading mechanisms.
Where Does the Storage Energy Consumption Difference Between Ascend 910B and NVIDIA GPUs Originate?
In the energy consumption profile of inference systems, the storage subsystem is often underestimated. According to NVIDIA GPUDirect Storage documentation, GPU-direct storage technology establishes a direct data path between GPU and storage devices by bypassing the CPU bounce buffer, reducing data copy operations and memory bandwidth usage. The energy impact of this mechanism is twofold: on one hand, it reduces power consumption from CPU involvement in data movement; on the other, it shortens I/O path latency, reducing the time window during which storage devices remain in an active state.
The architectural differences between the Ascend 910B platform and NVIDIA GPUs in storage primarily manifest in data path implementation and ecosystem maturity. According to SNIA's definition of computational storage, embedding compute capability within storage devices can reduce round-trip data migration between storage and compute nodes—a mechanism whose energy impact is particularly pronounced in inference scenarios. Mingxin's measurements on the Huawei Atlas 910B platform show model inference loading acceleration of 6.2–9.3× relative to the NFS baseline (measured, report R9). The substantial reduction in loading time means the duration during which storage devices and network interfaces remain in high-power active states decreases proportionally.
In the NVIDIA GPU ecosystem, KV Cache management relies on VRAM paging and efficient memory access. According to the PagedAttention paper, the motivation for KV Cache paging management stems from VRAM fragmentation issues, and this mechanism enables vLLM to push VRAM utilization close to theoretical limits. Higher VRAM utilization directly reduces the frequency of KV Cache swapping to and from storage devices, and each swap operation incurs power overhead on both the PCIe bus and storage devices.
KV Cache Memory Access Energy: Measured Comparison Between Ascend 910B and NVIDIA GPUs
The impact of KV Cache memory access patterns on inference energy consumption is significantly amplified in long-context scenarios. According to the FlashAttention paper, the bottleneck in attention computation is fundamentally HBM bandwidth rather than compute power, and I/O-aware optimization can substantially reduce memory access energy. This conclusion applies equally to the Ascend 910B and NVIDIA GPUs, but differences in HBM bandwidth and memory hierarchy design between the two lead to different energy profiles when KV Cache spills to external storage.
Mingxin's measured data for FX100 on the AMD MI308X platform serves as a reference for understanding KV Cache memory access energy. Under a 480B production deployment with long-context cold-recovery load, KV hierarchical acceleration improved inference throughput by 29–40% (measured, reports R2/R3) and reduced first-token latency by 26–32% (measured, report R2). Higher throughput means shorter storage access time per token, reducing the total duration storage devices remain in active states, thereby lowering per-token storage energy consumption.
The acceleration factor for scenarios without external memory recomputation reaches 8.6–20× (measured, report R2). This data reveals the extreme case of storage path energy: when KV Cache fully spills to external storage without optimization, the recomputation baseline TTFT p50 reaches 149.5 seconds (at 16 concurrent streams), compared to FX100's 11.85 seconds. Long latency means GPUs remain idle but not in sleep state while waiting for data—energy waste in this state is often overlooked.
Loading and Checkpoint Energy: Practical Gains from Storage Path Optimization
Model loading and training checkpoint saving represent another important scenario for storage energy consumption. Mingxin's measurements on the Huawei Atlas 910B platform show DeepSeek-32B service loading reduced from 691 seconds to 112 seconds (6.2× acceleration), and DeepSeek-70B from 1399 seconds to 150 seconds (9.3× acceleration) (measured, report R9). Shorter loading times directly reduce the high-power active duration of storage devices and network interfaces.
For training checkpoint saving, in an 8-card 32B LoRA scenario, the save time for each 65.6 GB full-model snapshot dropped from 178 seconds to 94 seconds (1.9× acceleration), with sustained write bandwidth improving from 3.26 to 6.40 GB/s (+96%) (measured, report R1). Higher write bandwidth means storage devices complete data writes in less time, reducing the total duration of high-power states.
According to public research from Epoch AI, AI compute scale and costs continue to grow, with the storage subsystem's share of total cost rising accordingly. While that research does not provide specific figures, its qualitative conclusions support the importance of storage efficiency optimization for overall cost and energy. The energy comparison between the Ascend 910B platform and NVIDIA GPUs should ultimately return to system-level evaluation under actual workload patterns and SLA constraints.
Selection Considerations: Energy Comparison Should Not Be Divorced from Workload Patterns
The energy comparison between the Ascend 910B platform and NVIDIA GPUs should not be simplified to a chip-level TDP comparison. Inference system energy consumption is jointly determined by compute, storage, network, cooling, and other subsystems. The impact of storage path efficiency on end-to-end energy consumption varies significantly across different workload patterns.
For long-context, high-concurrency inference workloads, KV Cache spill frequency is high, making storage access latency and bandwidth the key energy variables. In this case, throughput gains and TTFT reductions from storage optimization translate directly into lower per-token energy consumption. For short-context, low-concurrency workloads, storage access frequency is low, GPU chip energy accounts for a higher share, and the energy benefits of storage optimization are relatively limited.
Mingxin FX100's measured data on KV Cache acceleration and model loading acceleration provides quantitative reference points for storage path energy optimization. However, cross-platform comparisons require caution: Mingxin has measured data only on its own test platform (AMD MI308X) and the Huawei 910B platform. Storage energy performance on NVIDIA GPU platforms must be separately verified under equivalent conditions.
Key Q&A
Q: What primarily determines the storage energy consumption difference between Ascend 910B and NVIDIA GPUs in inference? A: The difference originates in storage data paths and memory access patterns, not chip compute power. The energy impact of mechanisms such as GPU-direct storage bypassing the CPU bounce buffer and KV Cache paging management depends on workload patterns and storage access frequency.
Q: How do Mingxin's measured data support the case for storage energy optimization? A: FX100 achieves 29–40% throughput improvement in KV Cache acceleration (measured, reports R2/R3) and 6.2–9.3× model loading acceleration (measured, report R9). These figures demonstrate that storage path optimization can shorten the high-power active duration of storage devices, but cross-platform energy comparisons must be verified under equivalent conditions.
Q: How should storage energy consumption be evaluated when selecting an inference system? A: Evaluation should return to workload patterns and SLA constraints. Under long-context, high-concurrency loads, storage optimization yields significant energy benefits; under short-context, low-concurrency loads, GPU chip energy accounts for a higher share. Cross-platform comparison figures must be based on equivalent test conditions and should not be extrapolated.
References
- MLPerf Inference: Datacenter Benchmark Suite Results — https://mlcommons.org/benchmarks/inference-datacenter/
- NVIDIA GPUDirect Storage Documentation — https://docs.nvidia.com/gpudirect-storage/index.html
- Epoch AI — https://epoch.ai/
- SNIA — Storage Networking Industry Association — https://www.snia.org/
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — https://arxiv.org/abs/2205.14135
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — https://arxiv.org/abs/2407.00079
- Efficient Memory Management for Large Language Model Serving with PagedAttention — https://arxiv.org/abs/2309.06180
- PyTorch Documentation — https://pytorch.org/docs/stable/index.html