SGLang Prefix Reuse and an External KV Tier: Do They Work Together
SGLang's RadixAttention reuses prefixes inside GPU memory — efficient but capacity-bound by HBM. External KV tiering solves "recover after eviction"; the two mechanisms complement each other. Mingxin's measurements are on the vLLM stack; SGLang adaptation will be backfilled by joint tests.
What RadixAttention solves — and what it doesn't
SGLang manages KV prefixes with a radix tree; hits inside GPU memory are reused directly, which is very efficient for high-prefix-repetition workloads (batch structured generation, multi-branch sampling) — excellent engine-level design (public-source basis).
But it faces the same physical constraint as vLLM prefix caching: HBM capacity. As session volume grows, tree nodes still get evicted, and revisits still mean re-compute. External tiering addresses exactly this segment — turning "evicted means lost" into "evicted but recoverable" (measured re-compute vs read-back gap in R2: TTFT p50 149.5 s vs 11.85 s).
Honest statement of measurement boundaries
Mingxin's current signed measurements (R1–R4) were done on the vLLM + LMCache stack. SGLang's tiered-cache direction (HiCache) plus an external tier is the same class of architecture, but we have no SGLang-platform numbers and will not extrapolate — that is this site's data discipline (no figures outside the four provenance classes).
What SGLang users can rely on: the FX array is a standard NVMe-oF block device that any engine's SSD sink layer can attach to; storage-tier bandwidth and sharing capabilities (measured anchors in R1/R3) are engine-independent. We welcome SGLang-platform joint tests under the G1–G4 gate process, with numbers backfilled from reports.
FAQ
Should I choose vLLM or SGLang?
Choose by workload: vLLM leads on general serving and ecosystem maturity; SGLang has design advantages for structured generation and multi-branch sampling. The external KV tier conflicts with neither.
Does SGLang have an official external-cache direction?
The SGLang community has a public tiered-cache evolution (HBM → RAM → storage); refer to their official docs for specifics — we do not speak for them.
Why not just publish SGLang performance numbers?
Because we have not measured them. This site's iron rule: only numbers with report IDs or reproducible calculations. Contact sales to schedule an SGLang joint test.
Data sources (verifiable)
Related reading
- vLLM + LMCache Integration in Practice: Versions, Builds and Pitfalls
- Prefix Caching: From In-HBM Reuse to a Cluster-Level Asset
- KV Cache Tiering: The Three-Level HBM → RAM → All-Flash Architecture
Joint test first, decisions second: gate-based acceptance with built-in stop-loss
The full costing model is provided as reproducible Python after NDA — customers can rerun it with their own parameters. Every key figure on this site carries a report ID and is open to third-party verification.
This site presents business-cooperation information and constitutes neither an investment offer nor any promise of returns. Measured data come from signed / official test reports (see the Evidence Library); vendor specs, public sources and estimates are labeled as such.