Managing Context Windows

Context windows are hard limits (4K-200K tokens) that agents must manage proactively

Token costs scale linearly with context size—efficient management saves money

Smaller, focused contexts improve speed, reduce latency, and enhance reasoning quality

Compression reduces verbosity while preserving information using LLM summarization or extractive methods

LLM-based compression offers best quality but costs API calls; extractive methods are faster

Compress old context first (e.g., messages older than 10 turns) to preserve recent relevance

Sliding windows maintain fixed-size context by dropping oldest messages as new ones arrive

Fixed-size windows keep last N messages; time-based windows keep messages from last T minutes

Hierarchical windows combine detailed recent context with compressed older summaries

System prompts must stay outside window management—never drop or compress them

Prioritization assigns scores based on recency, importance, and semantic relevance

Hybrid prioritization (40% recency + 60% importance) balances flow and critical facts

Tool results and user facts need high importance scores (0.8-0.9) to avoid premature dropping

Monitor what gets dropped—user complaints about "forgetting" indicate tuning needed

Combine compression, sliding windows, and prioritization for production-grade context management