The climb past human level is an engineering problem before it’s a philosophical one.
The paper’s claim is narrow and sharp: if AGI arrives, AI almost certainly does not stop there. It maps four ways the system keeps climbing and six things that could stall it. This page strips the argument to its load-bearing parts and translates each one into a decision your team is already making.
AGI, ASI, and the ceiling nobody can compute
The authors deliberately avoid sharp definitions and lean on a continuum instead. AGI is a system at roughly median-human level across most cognitive tasks. ASI is set high on purpose: a system that beats large, well-coordinated collectives of human experts across virtually every domain — think entire research fields or corporations, not a single genius. Narrow superhuman systems like AlphaFold or AlphaGo are explicitly ruled out.
The endpoint is Universal AI (AIXI) — provably optimal across all computable environments, and provably incomputable. It’s a ruler, not a roadmap: you can only approximate it from below with ever more capable systems.
Stop benchmarking against “a human.” Once you cross expert level on a task, human-comparison evals go silent. The product question becomes which collective does this replace — and that reframes pricing, positioning, and the eval suite all at once.
Effective compute is growing ~10× per year
Three independent trends compound. Multiply them and you get the headline number — an order of magnitude in effective compute, every year, sustained for over a decade. The authors call this a conservative estimate; some put it higher and accelerating.
1.5 × 2.5 × 3 ≈ 10× / year effective compute
The scaling thought experiment
Suppose base-model progress freezes the day AGI ships, but compute keeps climbing at 10×. You run 1,000 instances at first. Pure quantity does the rest.
Log scale. Or: one million instances running a hundred times faster. “Mere” scaling can buy capabilities that look qualitative.
Your cost-per-capability curve resets roughly annually. Anything you’re hand-optimizing today to fit a context window, a latency budget, or a unit-economics constraint may be free in 18 months — and a competitor who designed for the future curve eats the margin you defended. Architect for the model you can’t afford to run yet.
Digital minds don’t scale like human ones
The one fact that matters: we have the source code. Everything below follows from it, and every one of these gaps widens with more compute rather than saturating. Human-intelligence intuitions break here.
I/O speed
Ingests multiple books in seconds; with sensors and actuators, interacts with the world at ever-higher bandwidth.
Internal processing speed
“Thinking” scales with compute — deeper sequential or wider parallel. A major lever biology simply doesn’t have.
Working memory & memorization
Read/write bandwidth and capacity far past human limits; memorizing much of the internet is already demonstrated.
Substrate independence
Migrate between machines, even at runtime, even partially across heterogeneous hardware. Existence isn’t tied to one body.
Lossless replication
Copy not just the code (“DNA”) but the full memory state (“lifetime experience”). Backup, restore, spawn, halt, resume.
High-bandwidth experience sharing
Replay streams for training; among homogeneous instances, even raw gradients can be shared directly.
These are an architecture spec, not trivia. Stateless, checkpointable agents you can pause, fork, and replay turn advantages 04–06 into your scaling story: spin up ten thousand “experts,” snapshot the good runs, distill them back. Build the infra that treats an agent’s state as a first-class, copyable artifact.
Four pathways from AGI to ASI
These are not mutually exclusive. They run in parallel and can compound — not just add. Only the first has historical data to fit a forecast to; the rest are genuine unknowns.
Scale compute, models & data
Unknown: does more scale convert to broad new capability, or hit diminishing returns? Spiky or smooth?
Keep a live techno-economic model of cost vs. capability. Design so a 10× cheaper model is a config change, not a rewrite.
Algorithmic paradigm shifts
Unknown: true shifts (new architectures, optimizers, hardware) are unpredictable by nature and only show up at scale.
Decouple product surface from model internals. The team that can swap the engine in a sprint survives the next Mamba moment.
Recursive self-improvement
Unknown: AI accelerating AI R&D could explode (hyperbolic), fizzle fast, or anything between. No precedent to fit.
Automate your own R&D loop — eval generation, code, experiment triage — and track the AI-assisted share as a leading indicator.
Multi-agent group agency
Unknown: emergence in multi-agent systems is poorly understood. Markets? Borg-like collectives? Multi-agent scaling laws don’t exist yet.
Orchestration is core infra, not glue code: shared memory, message bus, mechanism design. Instrument group scaling — you’re collecting the missing data.
Even if each instance is merely human-level, a million coordinated copies running a hundred times faster is hard to call anything but a step change.
Six frictions that could stall the climb
The honest part of the paper: nobody knows whether these are speed bumps or walls. That ambiguity is itself the research program — and several of these frictions are things your release pipeline touches directly.
The data wall
Model size is outgrowing the supply of fresh high-quality text; exhaustion is estimated this decade.
Countered by: synthetic data, high-fidelity simulation, self-play, RL interaction data, distilled test-time search.
Economic & resource demand outruns supply
Investment, chips, energy, datacenter land, and rare earths may not scale fast enough to feed the current paradigm.
Countered by: rising AI-driven returns, efficiency gains, large-scale infra build-out.
The neural paradigm proves insufficient
Pretraining + post-training + scaffolding + test-time scaling may simply top out below the goal.
Countered by: continued evolution of the paradigm — much of it driven by sub-AGI AI itself.
Research gets harder
Low-hanging fruit gets picked; each gain costs more researchers, compute, and time (Bloom et al.: ideas get harder to find).
Countered by: 20× more “artificial researchers” is a compute decision, not a decade of hiring.
The abstraction barrier
Models trained on human concepts may be bounded by them — able to recombine, but not to discover genuinely new primitives from raw reality. Validating new concepts is gated by physical experiment speed (the “embodied bottleneck”).
Countered by: collective/multi-agent scaling even if individuals plateau; or a shift to grounded, interactive learning.
Deliberate slowdown & governance
Accidents, backlash, misuse, or regulation could cap capability scaling even where it’s technically feasible — compute-threshold licensing, mandatory evals, incident reporting.
Countered by: economic and military race dynamics; weak global coordination; regulatory arbitrage.
For progress to halt at human level, several of these would have to be hard walls at once. The authors’ low-confidence bet: more likely we plateau before AGI, or pass through to weak ASI relatively smoothly — with a real, non-dismissible chance of a fast recursive takeoff.
What even ASI cannot do
Superintelligence is not omniscience. ASI stays bound by limits that are precisely characterized — useful for resisting hype, less useful for predicting specific capabilities, since good approximations often dodge the worst-case bounds.
Don’t sell omnipotence and don’t buy it from a vendor. Any roadmap promising arbitrary physical or real-time outcomes (“cure aging by Q3,” “perfect weather simulation”) is selling past the floor. Ground capability claims in approximation quality, not theoretical limits — and remember the only reliable way to know how good an approximation is, is to run it.
The engineering playbook
Seven moves that follow directly from the paper, ranked roughly by how soon they pay off for a team shipping frontier AI.
Make evals a product, not a script
Public benchmarks (GPQA, SWE-bench, FrontierMath) are saturating. Build private, continuously refreshed, adversarial evals. Once you pass expert level, this is your only capability signal.
Treat proprietary interaction data as the moat
The data wall makes human text scarce. Every user interaction is a potential training signal. Instrument the loop: capture, curate, distill test-time search back into the model.
Turn inference compute into a product dial
Test-time scaling means “thinking harder” is a per-request budget, not a fixed cost. Expose reasoning tiers; cache and reuse traces; meter compute as a first-class resource.
Decouple product from model internals
A paradigm shift should be an engine swap, not a rebuild. Abstract the model behind a stable interface so you can adopt the next architecture in a sprint.
Automate and measure your own R&D loop
Recursive improvement starts weak and local. Track the AI-assisted share of code, experiments, and eval design — it’s your earliest read on whether takeoff dynamics are real for you.
Build orchestration as core infrastructure
If multi-agent is a path to capability, your message bus, shared memory, and coordination mechanisms are product, not plumbing. Instrument group scaling — the laws don’t exist yet, and you can write them.
Bake governance gates into the pipeline
Compute-threshold licensing, pre-deployment evals, and incident reporting are arriving. As you cut humans out of the loop, add corrigibility hooks, sandboxes, and kill switches by default — and log everything a regulator will ask for.
We can only see a short distance ahead, but we can see plenty there that needs to be done. — Turing (1950), the paper’s closing line