Short Communication: Token-Budget Policies for Long-Context LLMs
Abstract
We present Zipf-aware token trimming to reduce context size by 20% with negligible loss in answer quality.
Cite this article
Roberts, A. (2024). Short Communication: Token-Budget Policies for Long-Context LLMs. Research Explorations in Global Knowledge & Technology (REGKT), 3 (3). Retrieved from https://regkt.com/article.php?id=245&slug=short-communication-token-budget-policies-long-context-llms