on "context rot"
17 messages • Discord
thoughts on early discourse around the term "context rot" -> Codex (web)
- fjpaz
I don't think "rot" is the right word
https://bsky.app/profile/simonwillison.net/post/3lrw5nxrpkk2pWorkaccount2 on Hacker News just coined the term "context rot" to describe the thing where the quality of an LLM conversation drops as the context fills up with accumulated distractions and dead ends https://news.ycombinator.com/item?id=44308711#44310054
it's not like you left the context out for a week
implies that it happens on its own
when it's very much up to you whether things go that way or not
also
Right now I work around it by regularly making summaries of instances, and then spinning up a new instance with fresh context and feed in the summary of the previous instance.
this process literally elicits "context rot"
it's like pouring dirty water into a new cup, pouring in some fresh water and drinking it
"hygiene" / cleanliness is imo much more semantically apt
it just feels better/more agentic to aspire to cleanliness than to imply the issue is with the model
⸺
- fjpaz
I don't think "rot" is the right word
"context rot" is less of an issue for codex because I find that it's best used single-turn
but you can get better results from it to the degree that your task is well-specified
and if that specifying body of (con)text is large/complex then you ideally want to do that "hygienically" / in a way that is considerate of how each line will be received
and if that specifying body of (con)text is large/complex then you ideally want to do that "hygienically" / in a way that is considerate of how each line will be received
though at the same time
a significant chunk of the value offered by codex is in the fact that you don't have to do that
especially if the task isn't a complex one + is benefited by (or requires) dumping in 20-50k tokens of docs/logs, etc
⸺
- fjpaz
a significant chunk of the value offered by codex is in the fact that you don't have to do that