I benchmarked the same DeepSWE task session across every provider serving DeepSeek V4 Flash. I was originally researching how provider choice affects output quality and discovered how critical cache rate is to cost (observed a 17x difference).
A few findings:
- Token list price is completely dwarfed by cache % in the majority of cases when trying to optimize cost.
- Openrouter's listed cache benchmarks are useless since they are heavily biased by provider workload and their automatic "cheapest provider" routing correlates poorly with reality.
- Deepseek official is king, Most providers cache 94–99%. A few outliers: 61%, 55%, and 8%.
Hi all,
I benchmarked the same DeepSWE task session across every provider serving DeepSeek V4 Flash. I was originally researching how provider choice affects output quality and discovered how critical cache rate is to cost (observed a 17x difference).
A few findings:
- Token list price is completely dwarfed by cache % in the majority of cases when trying to optimize cost.
- Openrouter's listed cache benchmarks are useless since they are heavily biased by provider workload and their automatic "cheapest provider" routing correlates poorly with reality.
- Deepseek official is king, Most providers cache 94–99%. A few outliers: 61%, 55%, and 8%.