Why AGI Needs to Struggle

One of my favorite concepts is computational irreducibility

Computational irreducibility is an idea from Stephen Wolfram that says there are certain computational processes that cannot be simplified. The only way to attain an outcome is to actually go through each step of computation.

Weather systems and computational irreducibility

The atmosphere follows fluid dynamics equations, like the infamous Navier-Stokes equations, and chaotic interactions between temperature, pressure, and humidity. We use supercomputers to simulate the weather, but there is no shortcut to knowing the exact state of the atmosphere days or weeks ahead.

Tiny changes, like a butterfly flapping its wings, can lead to dramatically different outcomes. That is the butterfly effect in action. The only way to know the weather accurately is to run detailed simulations step by step. You cannot skip ahead. The system is computationally irreducible.

This is why long-term weather forecasts become unreliable. It is not that the math is flawed or that we are not smart enough. It is because nature forces you to go through every step. There is no hack. No compression trick. No simplified formula.

AI, trial and error, and the path to AGI

So here is the jump. As long as the steps have been processed, the solution is attained. That is the whole idea. There is no way around it.

I think about AI systems a lot. Right now, they give us answers instantly. Feels magical. But it is all based on content they have already seen or processed. And now we are building reasoning systems that try to go through problems step by step before giving an answer.

But here is the thing. From an epistemological perspective, Popperian philosophy says that knowledge is gained through trial and error. Nothing is ever certain. Only approximated. That is how humans improve. We get things wrong all the time. That is not a flaw, it is a feature. If we ever reached a perfect, final answer, there would be no need to keep thinking. But that has never happened. We always just keep refining.

That is how cars got faster, computers got smarter, technology moved forward. Each improvement came from someone trying, failing, tweaking, then trying again. Iteration is the engine.

And yet we treat AI like it should just know everything already. As if true intelligence means giving perfect answers on the first try. But it is not. Real intelligence comes from engaging with reality, failing, adapting, and improving. It is a process. Always a process.

Even if you gave AI all the data in the world, it would not just spit out perfect original solutions. Because having data is not enough. Intelligence is not about storing information. It is about interacting with the world. Making predictions. Testing them. Learning from the outcomes.

Take software development. I can ask AI to generate a game, and it might give me something decent. But then I have to iterate. Prompt again. Refine. Adjust. Why? Because I do not know the perfect solution either. I am figuring it out too. Just like AI should be doing. That is why we have software versions. Constant improvement. Never done.

If we want AI to reach human-level intelligence, we have to design it to go through the same kind of process. It needs memory. It needs iteration. It needs experience. That is how it learns. That is how it grows.

Superintelligence might be a separate discussion. But for AI to get to real AGI, it has to walk the path. It has to make mistakes. Try things. Get better. Just like us.

Iterate or stagnate. That is the reality.

AGI, for me, would be something that can go off and improve based on experience. Not because it was told what the answer is, but because it figured it out.

And maybe that is the deeper truth. In Wolfram’s Ruliad, the theoretical space of all possible rules and all possible paths, everything that can be computed exists somewhere. But there is no skipping ahead. No fast lane. To reach meaningful knowledge, to grow intelligence, you have to walk the path. Step by step. Rule by rule. Because even in a universe of infinite possibilities, the only way forward is through.