The risk of KV cache compression
paper-conference
Minimax risk and design principles for accurate KV-cache compression under causal masking.
Resources
Publication history
Preprint: 1 July 2026. Accepted at NeurIPS 2026; final proceedings details are forthcoming.
Summary
The paper characterizes the minimax risk of KV-cache compression through the intrinsic compressibility of the cache. This analysis identifies when accurate compression is possible and yields design principles for causal attention that support efficient prefill and autoregressive decoding. A practical algorithm instantiates these principles, with targeted LongBench experiments assessing its performance.
Citation
@inproceedings{haverbeck2026kvcacherisk,
author = {Haverbeck, Lukas and Amo Alonso, Carmen and Posada-Moreno, Andres Felipe and Trimpe, Sebastian and Pavone, Marco},
title = {The risk of KV cache compression},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
note = {Accepted at NeurIPS 2026; proceedings forthcoming},
eprint = {2607.01520},
archivePrefix = {arXiv},
url = {https://neurips.cc/virtual/2026/poster/153754}
}