The limits of intelligence · part 4 of 4 · by Bill Guan
Even a superintelligence has to wait for some answers
The big question
What could a superintelligence still not do?
Being smarter buys an enormous amount, and some limits stay put anyway because they're built into nature and mathematics. You're about to meet four walls no mind can climb, and see why every one of them sits far above us.
In 1814 the mathematician Pierre-Simon Laplace imagined the ultimate know-it-all. "An intelligence knowing all the forces acting in nature at a given instant, as well as the momentary positions of all things in the universe," he wrote, "would be able to comprehend in one single formula the motions of the largest bodies as well as the lightest atoms in the world" 1. For a mind like that, now known as Laplace's demon, the future would be as easy to read as the past, and a superintelligence is the closest thing to it anyone has seriously tried to build.
Your guess first
Laplace's demon would track the planets perfectly. If you get the Earth's position wrong by just 15 meters today, how far off is your prediction 100 million years from now?
About 150 million kilometers, roughly the whole distance from the Earth to the Sun. Tiny errors grow until they swallow the answer, and no amount of intelligence gets around that.
Wall 1: chaos turns tiny errors into total ignorance
The astronomer Jacques Laskar worked out how fast that happens to the Earth's own orbit: "an error of 15 meters in the initial position of the Earth gives rise to an error of about 150 meters after 10 Ma, but the same error becomes 150 million km after 100 Ma" 2, where Ma means millions of years. The error grows roughly tenfold every 10 million years, so past about 100 million years no amount of computing can tell you where the Earth will be.
The air above us does the same thing on a much shorter clock. In 1963 the meteorologist Edward Lorenz showed that in a simple model of moving air, two almost identical starting points drift apart until the runs look nothing alike. His conclusion still stands: for something like the atmosphere, "prediction of the sufficiently distant future is impossible by any method, unless the present conditions are known exactly" 3.
Tiny errors in where you start grow until they swamp the forecast. A 2019 study by Fuqing Zhang and colleagues put numbers on it for the weather. Today's useful forecasts reach "around 10 days," and "the ultimate limit of midlatitude day-to-day weather predictability is about 2 weeks." Even cutting the uncertainty in today's measurements tenfold buys only "up to 5 days" more 4.
To be fair, that limit is being tested right now. A 2026 preprint (a study shared before peer review) using an AI weather model with carefully tuned starting conditions reports forecast skill "lasting beyond 30 days," though its authors say it's still an open question whether such starting conditions can be found in real time 5.
A superintelligence would forecast better than we do, maybe much better, and yet somewhere out past the edge of chaos the details are gone for any mind, because the starting information was never there to begin with.
Wall 2: some programs can't be predicted at all
In 1936 Alan Turing proved something that still surprises people: there is no general method that can look at any program and decide whether it will eventually stop or run forever. In modern terms the question is called the halting problem, and "the halting problem is unsolvable" 6. That holds for any mind that computes, however fast.
My favorite illustration is a contest called the Busy Beaver. Take every possible tiny computer program of a given size and ask which of the ones that eventually stop runs the longest. For programs with five instructions, an online collaboration finished the search in July 2024 7 and proved the answer with a computer-checked proof: 47,176,870 steps 8. Getting there meant checking more than 181 million machines.
For six instructions, the answer may never be known. One six-instruction machine, nicknamed Antihydra, behaves like a famous unsolved math puzzle known as the Collatz problem, and settling it means solving "a Collatz-like problem" 9. The Busy Beaver numbers grow faster than any function a computer can calculate 10, so every new step up eventually outruns every mind.
Wall 3: some true things can't be proved
Five years before Turing, Kurt Gödel found a deeper crack. Take any consistent system of rules F (consistent meaning it never proves a contradiction) that is powerful enough to do arithmetic. Gödel showed that "there are statements of the language of F which can neither be proved nor disproved in F," and that such a system "cannot prove that the system itself is consistent" 11.
Mathematics is built from axioms, the starting rules everyone agrees to assume, up to theorems, with conjectures sitting off to the side as statements nobody has proved yet. Gödel guarantees that the side pile never empties. A superintelligence would prove a staggering number of theorems, and it would still face true statements its rules can't reach.
Wall 4: some answers have no shortcut
The last wall is subtler. Some processes, Stephen Wolfram argues, are "computationally irreducible": "there can be no way to predict how the system will behave except by going through almost as many steps of computation as the evolution of the system itself" 12. The only way to see the end is to run every step. How common that is remains Wolfram's conjecture, but if he's right, a lot of the world can only be watched step by step as it unfolds.
A related open question asks: "If it is easy to check that a solution to a problem is correct, is it also easy to solve the problem?" 13. Mathematicians call it P versus NP, and it's one of seven million-dollar Millennium Prize problems. Nobody knows the answer, and most of the cryptography protecting your bank account rests on unproven bets that some problems really are hard 14.
No single mind is the best at everything
Even learning has limits. The "No Free Lunch" theorems show that, averaged over every possible problem, "the average performance of any pair of algorithms across all possible problems is identical" 15. A mind that's brilliant on some problems pays for it on others. Our universe has structure, so real learners can still be very good at real problems, and what the theorems rule out is a universal genius that wins everywhere.
The closest thing to a perfect learner on paper is a design called AIXI, proposed by Marcus Hutter as an agent that "behaves optimally in any computable environment." Its catch, in Hutter's own words: "The major drawback of the AIXI model is that it is incomputable" 16.
We hit the same walls from much further down
All of these walls are real, and all of them sit far above us. Humans can't forecast the weather three weeks out either, and we can't settle the six-instruction Busy Beaver. A mind can be vastly smarter than us without breaking any of these limits, the way a chess engine beats every grandmaster without ever solving chess.
My assessment: I find that oddly comforting about the universe itself. However smart minds get, there will always be true statements with no proof, and futures you can only learn by waiting for them to happen.
Self-copying probes could carry one set of values to billions of galaxies, so whoever launches first might shape everything they reach.
Sources
- Pierre-Simon Laplace, A Philosophical Essay on Probabilities (1814), as translated in Carl Hoefer, "Causal Determinism," Stanford Encyclopedia of Philosophy (rev. 2023). plato.stanford.edu/entries/determinism-causal/
- Jacques Laskar, "Is the Solar System stable?" (2012). arxiv.org/abs/1209.5996
- Edward Lorenz, "Deterministic Nonperiodic Flow," Journal of the Atmospheric Sciences 20, 130 (1963). math.bu.edu/people/mabeck/Fall14/Lorenz63.pdf
- Fuqing Zhang et al., "What Is the Predictability Limit of Midlatitude Weather?" Journal of the Atmospheric Sciences 76, 1077 (2019). texmex.mit.edu/ftp/pub/emanuel/PAPERS/Zhang_etal_2019.pdf
- Vonich and Hakim (2026 preprint, v2). arxiv.org/abs/2504.20238 [TBD: full title; quote came via a fetch summary, medium confidence]
- Neil Immerman, "Computability and Complexity," Stanford Encyclopedia of Philosophy. plato.stanford.edu/entries/computability/
- bbchallenge, "The story." bbchallenge.org/story
- The bbchallenge Collaboration, "Determination of the fifth Busy Beaver value" (2025). arxiv.org/abs/2509.12337
- bbchallenge wiki, "BB(6)"; and bbchallenge, "Antihydra." wiki.bbchallenge.org/wiki/BB(6) and bbchallenge.org/antihydra
- Scott Aaronson, "The Busy Beaver Frontier" (2020), Proposition 2. www.scottaaronson.com/papers/bb.pdf
- "Gödel's Incompleteness Theorems," Stanford Encyclopedia of Philosophy (rev. 2025). plato.stanford.edu/entries/goedel-incompleteness/
- Stephen Wolfram, A New Kind of Science (2002), p. 739. files.wolframcdn.com/pub/www.wolframscience.com/nks/nks-ch12…
- Clay Mathematics Institute, "P vs NP." www.claymath.org/millennium/p-vs-np/
- Sanjeev Arora and Boaz Barak, Computational Complexity (draft, 2007), cryptography chapter. theory.cs.princeton.edu/complexity/cryptochap.pdf
- David Wolpert and William Macready, "No Free Lunch Theorems for Optimization," IEEE Transactions on Evolutionary Computation 1, 67 (1997). www.cs.ubc.ca/~hutter/earg/papers07/00585893.pdf
- Marcus Hutter, Universal Artificial Intelligence (Springer, 2005). www.hutter1.net/ai/uaibook.htm