vLLM TorchTPU: Zero-copy KV cache offloading and disaggregated serving

(vllm.ai)

1 points | by mikemo88 5 hours ago ago