Intelligence is not a straight line. From the 1943 neural switch to today’s scaling limits, Hitting
Walls chronicles the structural
walls AI has hit—and the radical
ladders built to bypass them.
Most AI histories trace the victories. This one maps the
wreckage. A guide to understanding
the present by deconstructing the past.
A history of artificial intelligence told through its failures—where confident theories broke, and what those breaks reveal about intelligence, learning, and generality.
W
e like to tell the history of science as a clean ascent. A relay race of geniuses handing off the torch. But this is the classic error of the survivor.
In war, we look at the planes that return, count the bullet holes in the wings, and decide to armor the wings. We forget that the planes hit in the engine never came back to be counted.
This book is about the planes that didn't come back.
Artificial Intelligence did not advance by piling success on top of success. It advanced by crashing. From the Biological Wall of 1943 to the Epistemic Wall of today, every breakthrough was born from a specific, catastrophic failure. To understand the machine you are using today, you cannot just study the winner. You have to study the wreckage that made the winner necessary.
Trace the lineage of machine logic. Each BOXED NODE is a Ladder that scaled the red Wall beneath it—only to hit its own limits at the Wall ahead. The blue labels mark the Dimensions where the tribes diverged before finally converging into Tokens. Trace the history of the climb.
There was a single dream: to copy the brain. But the early models hit wall after wall. The biological neuron was too Rigid, the Perceptron too Linear, and the first deep networks too hard to train. It took a 30-year "Winter" of isolation to build the mathematical ladder out of this hole.
To survive the walls of Space, Time, and Complexity, AI had to fracture into specialized tribes. Each architecture was a specific ladder built to scale a specific wall.
The walls crumbled when we realized that images, words, and actions were all just Tokens. The Transformer unified the tribes, but it did not solve the puzzle—it merely industrialised it.
Hover over the nodes to trace the lineage. Pan and tap the nodes to trace the lineage.
Intelligence wasn't built in a straight line. It was built by hitting walls at full speed.
The founders of deep learning were ignored for decades. They survived on conviction alone.
Language, Vision, and Action have finally merged. The result is the modern mind.
We have not reached the summit. We are just at the foot of the Energy, Plasticity, and Epistemic walls.
A structural post-mortem in five acts. Tracing the moments where the physics of the architecture forced the field to reroute—from the first rigid gates to the unified fever dreams of today.
Walter Pitts & Warren McCulloch attempt to map the soul to logic.
Frank Rosenblatt's Perceptron hits the first mathematical wall.
The "Conspiracy" of Hinton, LeCun, and Bengio keeps the flame alive.
Yann LeCun teaches a machine to see (CNNs).
Schmidhuber & Hochreiter solve the memory problem (LSTMs).
Scale meets Data: AlexNet conquers ImageNet.
AlphaGo plays Move 37 in Seoul.
The Transformer shatters the sequential wall.
We built a machine that can dream, but forgot to teach it to wake up.
History was rarely made by the comfortable, but by the outcasts who stayed near the wreckage long enough to understand it. These are the architects who refused to negotiate with the wall.
A homeless teenager and a philosopher mapping the brain on a kitchen table.
Promising the world a machine that could see—right before the math hit back.
The back-injured professor and the "conspiracy" that stayed near the wreckage for decades.
The video game designer and the scale-engineer who realized that to build a mind, you first have to build an engine.