CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents

Problem

The paper addresses a gap in the capability of long-horizon coding agents, specifically the need for effective compaction across sessions due to limited context windows. This issue is particularly relevant for applications requiring efficient resource management in coding tasks. The work is presented as a preprint and has not yet undergone peer review.

Method

The authors propose a novel technique called CliffCompaction, which achieves up to 50% cost reduction in context management. The method operates by maintaining the fidelity of compacted information through truncation or content dropping, avoiding rephrasing or rewriting. This approach ensures that the essential information is preserved while reducing the computational burden. The implementation is provided as an open-source, scaffold-agnostic API-proxy, compatible with various coding harnesses, including Claude Code and Codex.

Performance metrics are evaluated using two benchmarks: Terminal-Bench and KernelBench. In Terminal-Bench, CliffCompaction demonstrates an improvement of over 10 percentage points compared to the cost of two full-context runs. For KernelBench, the method achieves CUDA kernel speedups of 2.23x after 200 steps and 3.58x after 400 steps, outperforming specialized search algorithms and trained agents, respectively.

Results

• Cost Efficiency: Up to 50% reduction in costs compared to standard context management.
• Terminal-Bench Improvement: Over 10 percentage points improvement against the cost of two full-context runs.
• KernelBench Speedup: 2.23x after 200 steps compared to specialized search algorithms.
• KernelBench Speedup: 3.58x after 400 steps compared to trained agents.

Limitations

The authors do not report any limitations in the study. However, the absence of reported limitations may suggest a need for further validation in diverse contexts or with varying datasets.

Why it matters

CliffCompaction has significant implications for the development of long-horizon coding agents, particularly in resource-constrained environments. By enabling efficient compaction, this method can enhance the performance of coding tasks while reducing computational costs, paving the way for more scalable and effective AI applications in various domains.