Predictive Speculative KV Replication for Bursty LLM Inference

(jwlabs.vercel.app)

33 points | by shreybirmiwal 10 hours ago ago

3 comments

$msp26 7 hours ago

the hand drawn diagrams and highlights are charming
[-]
- $shreybirmiwal 5 hours ago
  
  thank you so much
$holtkam2 4 hours ago

Awesome animations!