HBM Efficiency and KV Tiering Practice on MetaX GPUs
MetaX and other domestic GPUs have limited HBM, making an external KV tier more valuable. R5 completed a seven-way comparison on a single N260, arguing that the external tier can substitute for 128 GB-class HBM residency.
The HBM constraint of domestic GPUs
Domestic GPUs generally carry less HBM per card than international flagships, so KV eviction under long-context concurrency happens more often — on these platforms external tiering is a necessity, not a nicety. That is why Mingxin chose the MetaX N260 for the dedicated HBM-efficiency measurements.
What the R5 seven-way comparison shows
A 14B model across seven comparison groups (GDS direct read, two re-compute HBM tiers, low-HBM direct read and more), with a three-step argument concluding that an external KV tier can substitute for 128 GB-class HBM residency. The GDS direct-read path was verified working on MetaX. The methodology is portable; the numbers belong to that platform (labeled honestly, not mixed with MI308X).
What it means for procurement: when HBM runs short, price the external tier first — all-flash at ≈ ¥2,014/TB (FX100 fully-populated basis) differs from HBM by orders of magnitude.
FAQ
Is the MetaX software stack mature?
R5 ran all seven comparison groups on the MetaX stack, including the GDS direct-read path; versions and adaptation notes are in the report (Evidence Library download).
What about other domestic GPUs like Hygon or Moore Threads?
The methodology is platform-independent; measured platforms so far are MetaX/Ascend/AMD (R5/R9/R1–R4). Other platforms proceed on joint-test schedules with no numbers pre-promised.
Do single-GPU measurements generalize to clusters?
R5 focuses on the HBM-efficiency methodology (arguable on a single GPU); for cluster-level KV tiering throughput see R2/R3 (MI308X platform) — MetaX cluster measurements await backfill.
Data sources (verifiable)
Related reading
- When GPU Memory Runs Out: The HBM-Equivalence Methodology of an External KV Tier
- Storage Acceleration for AMD MI308X Inference Clusters
- KV Cache Tiering: The Three-Level HBM → RAM → All-Flash Architecture
Joint test first, decisions second: gate-based acceptance with built-in stop-loss
The full costing model is provided as reproducible Python after NDA — customers can rerun it with their own parameters. Every key figure on this site carries a report ID and is open to third-party verification.
This site presents business-cooperation information and constitutes neither an investment offer nor any promise of returns. Measured data come from signed / official test reports (see the Evidence Library); vendor specs, public sources and estimates are labeled as such.