vLLM + LMCache Integration in Practice: Versions, Builds and Pitfalls
vLLM connects to an external KV tier through the LMCache connector; on ROCm this needs source-build adaptation. Mingxin's measured stack is vLLM 0.20.1+rocm721 + LMCache upstream mainline plus the parallel-read patch (TTFT 4.1×).
Integration path and version matrix
The measured combination: ROCm 7.2, vLLM 0.20.1+rocm721, LMCache upstream mainline (built from source, 2026-06-29), Qwen3-Coder-480B-FP8 (TP=8) — versions and conditions published in full (R1–R4); copy them to reproduce.
On CUDA, pip install works; on ROCm you build from source and handle several platform differences. During the build Mingxin isolated and fixed an fs:// storage-backend defect (recorded in R1/R8); the patch ships with the export package.
Two unavoidable pitfalls and their fixes
Pitfall one: default serial disk reads. LMCache cold reads default to block-by-block serial I/O and cannot saturate the array — the parallel-read patch lifted single-GPU concurrency-16 cold-read bandwidth from 0.98 GB/s to 5.23 GB/s and TTFT from 37.97 s to 9.30 s (4.1×, measured, R1).
Pitfall two: underestimating the storage tier. A local single drive tops out under high concurrency; the external 4-drive RAID0 array (over 100 GbE RoCEv2) actually measured 26–32% lower TTFT (R2). Integrate and joint-test in the production network form from the start.
FAQ
How long does integration from scratch take?
On CUDA, days by the official docs; on ROCm we recommend Mingxin's build adaptation layer and deployment scripts (delivered during the joint test, incl. the R8 export package) to avoid re-treading the pits.
Will vLLM upgrades break the integration?
LMCache attaches through the connector interface, weakly coupled to vLLM; Mingxin provides regression scripts — rerun the baseline (G2 basis: R1 ±10%) before switching production.
Can I validate the software stack before buying hardware?
Yes: LMCache is open source and functional validation runs on local drives; the performance gap (local drive vs external array, TTFT 26–32%) is in the R2 control-group data.
Data sources (verifiable)
Related reading
- Productionizing LMCache: From Open-Source Component to Signed Benchmarks
- ROCm Inference Stack Engineering Guide: Productionizing vLLM/LMCache on AMD
- KV Cache Offloading: Putting the Inference Cache on an All-Flash Array
Joint test first, decisions second: gate-based acceptance with built-in stop-loss
The full costing model is provided as reproducible Python after NDA — customers can rerun it with their own parameters. Every key figure on this site carries a report ID and is open to third-party verification.
This site presents business-cooperation information and constitutes neither an investment offer nor any promise of returns. Measured data come from signed / official test reports (see the Evidence Library); vendor specs, public sources and estimates are labeled as such.