Context rot: why your agent gets dumber as the session grows

The term came out of practitioner forums and stuck because everyone recognized the symptom: as a session grows, the model starts forgetting constraints, re-asking settled questions, and undoing its own work. A study by the team at Chroma tested the effect across 18 models and found the same pattern everywhere — performance degrades as input length grows, even on tasks the model handles easily at short lengths.

Anthropic's context engineering essayexplains the mechanism: models have an attention budget, and every added token competes in an n² web of pairwise attention. Their guiding principle is worth pinning above your desk: find the smallest set of high-signal tokens that maximizes the outcome you want. Andrej Karpathy's reframe — context engineering over prompt engineering — is the same idea from the other end: the craft isn't writing one clever prompt, it's curating what fills the window at each step.

1. Put critical rules in the first thousand tokens. Position matters more as context grows. Google's long-context docs and Anthropic's both converge on the same placement rule for big inputs: bulk material first, the actual query and critical instructions at the end — Anthropic measured up to a 30% quality difference from placement alone.

2. Clear on topic change; compact on phase change. Switching tasks inside one long session drags the old task's residue into the new one's attention budget. New topic → new session. Same topic but a finished phase → compact before degradation starts, not after.

3. After two failed corrections, restart with a better prompt. Anthropic's Claude Code guidance is blunt: a clean session with a better prompt almost always beats a long session with accumulated corrections. The instinct to "push through" is exactly backwards — every failed correction is more rot.

4. Load task material just-in-time instead of always-on. Standing instructions belong in a lean rules file (see the 300-token rule). Task prompts — review checklists, debugging procedures, planning templates — belong in a prompt library and enter the window only when invoked:

# nothing in standing context; loaded only for this task, at full strength:
ple get root-cause-analysis --var error_and_logs="$(tail -n 80 app.log)" | claude -p

The uncomfortable implication of the attention budget: your best sessions are short ones that start sharp. That only works if starting sharp is cheap — which is the real argument for keeping prompts versioned and one command away instead of scattered across old chats. Restarting costs nothing when the prompt that starts the next session is better than the one that started the last, and its version history is the record of how it got that way.