Why AI forgets — the recall wall, explained simply
Ask a person to read a long letter and they will miss a few details. Ask most efficient AI models to do the same and they will miss them predictably — not because they are careless, but because of how they are built.
A language model reads text one piece at a time, and as it reads it must keep some record of what came before. There are two classical ways to do this. The first is to keep everything: every word stays available, and when the model needs a fact it looks back at the original. This is what full attention does. It remembers perfectly — and the cost of looking back grows with the square of the length of what it has read. Double the document, quadruple the bill.
The second way is to compress: squeeze the past into a fixed-size summary as you go. This is what fixed-state models do. The cost stays flat no matter how long the text grows. But a fixed-size summary can only hold so much. Once the text carries more facts than the summary has room for, something must be thrown away. The model does not choose badly — it simply cannot keep everything. This is the recall wall, and it is provable, not anecdotal: in controlled tests, models that compress lose facts they read moments earlier, while models that keep everything recall them almost perfectly.
So every system today picks a side: remember everything and pay quadratically, or pay little and forget. But the wall is not a law of nature — it is a consequence of the fixed-size summary. Remove that constraint, keep the past exact, and learn to read only the parts that matter, and the wall stops being load-bearing. That is the work.