Gated Joint Testing: 128 Cards to Thousands

As AI computing demand leaps from hundreds of cards to thousands or even larger scales, the construction and scaling of computing centers are undergoing a paradigm shift from "technical validation" to "mass production." The traditional model of "one-time procurement, post-deployment optimization" reveals significant drawbacks in the face of high hardware investment and complex software stacks: long cycles, high risks, and unpredictable Total Cost of Ownership (TCO). A joint testing model based on clear performance gates and phased validation is becoming a key choice for leading enterprises and service providers building robust computing foundations. Its core value lies in transforming the uncertainty of large-scale investment into a series of measurable, verifiable, and stoppable technical milestones, thereby front-loading risks and making TCO controllable.

Measurable Performance: Why are "Gates" More Reliable than "Promises"?

In computing infrastructure construction, there is often a gap between a vendor's performance "promises" and the "measured" results in the final production environment. This gap stems from differences between test and production workloads, the complexity of software stacks, and hidden bottlenecks arising from multi-component coupling. The core of gated joint testing is to transform vague promises into a series of specific, repeatable performance indicators (KPIs), which serve as "checkpoints" for project progression.

Taking large-scale model inference as an example, key bottlenecks often lie in the storage and access efficiency of intermediate states like KV Cache. In joint testing practices with clients, Mingxin Technology sets clear gate metrics. For instance, in measured, report R2, for a long-context inference scenario with a 480B parameter model, the core gates were set as "First Token Latency (TTFT) reduction ≥ 25%" and "Inference throughput increase ≥ 29%". Measured data shows that after deploying the FX100 all-flash array, TTFT p50 was reduced by 26-32%, and throughput increased by 40% at the optimal operating point (measured, report R2). These metrics, based on real workloads and reproduced in the client's environment, are far more convincing than paper specifications or data from ideal lab conditions.

More importantly, the gates are phased. From equipment arrival acceptance (G1), establishing single-machine baseline performance (G2), to core performance gate validation (G3), and finally a 72-hour stability stress test (G4), each step has clear pass/fail criteria. This model ensures problems at each stage can be identified and located promptly, preventing issues from accumulating until the final integration phase, which could cause project delays and cost overruns.

Controllable TCO: How Does Front-loaded Validation Optimize Full-cycle Costs?

The TCO of a computing center includes not only the capital expenditure (CapEx) for hardware procurement but also operational expenditures (OpEx) such as deployment, debugging, maintenance, energy consumption, and business losses due to underperforming hardware. Gated joint testing optimizes both cost components by front-loading risks.

On the CapEx side, the joint testing model is often combined with a "stop-loss if standards are not met" clause. This means if equipment fails to meet agreed performance at core gates (e.g., G3), the client has the right to terminate cooperation, avoiding the sunk cost of deploying unsuitable hardware into production. Mingxin Technology's approximately 10-week gated joint testing cooperation model institutionalizes this risk control mechanism.

On the OpEx side, performance certainty directly translates to business efficiency and resource utilization. For example, during model service loading, slow storage severely hampers service restart and elastic scaling speeds. Measured, report R9 shows that on the Huawei Atlas 910B platform, compared to a traditional Network File System (NFS), using the FX100 array reduced the service loading time for DeepSeek-32B from 691 seconds to 112 seconds, a 6.2x acceleration; for DeepSeek-70B, the acceleration ratio reached 9.3x. This deterministic acceleration capability makes compute cluster scheduling more flexible and resource utilization higher, indirectly lowering the operational cost per unit of computing power.

Furthermore, in AI training scenarios, the speed of saving and reading Checkpoints directly impacts training task fault tolerance and iteration efficiency. Measured, report R1 indicates that in LoRA training of an 8-card 32B parameter model, the FX100 reduced the save time for each 65.6GB model snapshot from 178 seconds to 94 seconds, a 96% bandwidth improvement. This reduces training pauses caused by Checkpoint operations, increasing the productive time for researchers and expensive computing resources.

From Hundreds to Thousands of Cards: How Does the Gated Path Ensure Smooth Scaling?

Scaling from 128 cards to thousands is not a simple matter of adding hardware. It involves network topology restructuring, challenges in achieving linear storage bandwidth growth, multi-task resource isolation, and an exponential increase in global management complexity. The gated construction path provides a reproducible "blueprint" for this kind of large-scale scaling.

First, gated joint testing at the scale of hundreds of cards validates not only single-device performance but also the collaborative working mode and performance baseline of the entire technology stack (compute, storage, network, software). For example, the collaborative baseline established between 8-card MI308X and FX100 arrays in the measured, report R1/R4 platform, along with its test scripts, performance data, and tuning parameters, can be packaged as a standardized template. When scaling to thousands of cards, this template can be replicated and extended as a unit, significantly reducing the effort and uncertainty of adaptation and tuning from scratch.

Second, storage system scalability becomes critical for thousand-card clusters. Mingxin's FX product line, from the PCIe 3.0 FX100 to the PCIe 6.0 FX400 (expected mass production by end of 2026), provides a smooth upgrade path for bandwidth and IOPS. The NVMe-oF (e.g., RoCEv2) architecture validated in hundred-card joint testing, with its low latency and high bandwidth characteristics, lays the foundation for storage pooling and unified namespace implementation in thousand-card environments. The LMCache parallel read patch validated in measured, report R1, which improved TTFT by 4.1x in a single-card concurrency scenario, is a software-level optimization that can also be scaled to larger clusters to alleviate memory wall pressure.

Finally, stability gates (like the 72-hour stress test in G4) are particularly important during scaling. In large-scale clusters, any minor instability can be amplified, leading to entire job failures. Front-loaded, rigorous stability testing helps expose potential issues in hardware, drivers, firmware, or network configuration early, ensuring high availability in the scaled production environment.

Conclusion

The construction of computing centers is evolving from crude resource accumulation to refined efficiency operation. The gated joint testing model, with its characteristics of being measurable, verifiable, and stoppable, provides deterministic assurance for computing investments. It serves as a robust bridge connecting innovative technologies (like high-performance storage acceleration) with mass production applications. Based on the FX series storage acceleration devices and the gated cooperation model, Mingxin Technology is committed to jointly validating performance improvements across all scenarios—from model inference and training to large-scale service deployment—with clients, providing a measured and tested path choice for building efficient and controllable next-generation computing infrastructure.

Key Q&A

Q: What are the main advantages of gated joint testing compared to traditional procurement models? A: The main advantages are risk control and TCO optimization. It transforms performance promises into phased, verifiable measured gates (e.g., TTFT reduction ≥ 25%) and is paired with a "stop-loss if standards are not met" mechanism to avoid sunk investments. Simultaneously, front-loaded performance and stability validation (e.g., 72-hour stress tests) can significantly reduce later-stage operational risks and costs.

Q: In AI computing scenarios, what specific metrics do storage performance gates typically focus on? A: Core metrics typically focus on bottleneck points in the model lifecycle. These include: 1) Inference First Token Latency (TTFT): e.g., FX100 reduced TTFT for a 480B model by 26-32% in measured tests (measured, report R2); 2) Inference Throughput: measured increase of 29-40% (measured, report R2/R3); 3) Model Loading Time: acceleration of 6.2-9.3x compared to NFS (measured, report R9); 4) Training Checkpoint Save Bandwidth: measured improvement of 96% (measured, report R1).

Q: How does the gated path assist in scaling from validation at the hundred-card level to thousands of cards? A: It provides a reproducible scaling blueprint. The technology stack collaboration templates, performance baselines, and software optimizations (e.g., the LMCache patch) validated at the hundred-card scale can be directly reused. Meanwhile, the validated NVMe-oF storage architecture ensures the scalability of storage pooling, and the front-loaded, rigorous stability testing can expose potential issues in large-scale deployments early, ensuring smooth scaling.