Thirty researchers — including Yann LeCun, Azalia Mirhoseini, Jason Cong, and Christos Kozyrakis — just published a paper called AI+HW 2035. The title is dry. The ambition is not. They want to make AI a thousand times more efficient in the next decade.
Not a thousand times faster. A thousand times more efficient. That distinction matters more than anything else in the paper.
The number and why it is not crazy
A thousand feels like the kind of number someone writes on a whiteboard during a brainstorm and then quietly erases before the adults arrive. But the paper breaks it down multiplicatively, and that changes the math entirely.
Ten times from algorithms — sparse computing, mixed-precision arithmetic, models that learn to skip the work they do not need. Ten times from hardware — domain-specific accelerators that do one thing well instead of everything passably. Ten times from system architecture — fixing the memory wall, reducing data movement, treating interconnects as first-class citizens instead of afterthoughts. Ten times from application-level optimization — eliminating the astonishing amount of redundant computation that nobody notices because GPUs are fast enough to hide it.
Multiply those together and you get ten thousand.
The paper calls a thousand “conservative but achievable.” Which is either the confidence of people who have done the math or the optimism of people writing a vision document. Probably both.
The real argument
The surface argument is about efficiency. The deeper argument is about physics.
Current AI training consumes gigawatt-scale power. That is not a metaphor. Training a frontier model requires the sustained electrical output of a small city. The trajectory is unsustainable in the most literal sense of the word — not unsustainable like “this seems like a lot,” but unsustainable like “there are not enough power plants.”
The paper’s position is blunt: the future of AI depends not on scaling intelligence, but on scaling efficiency. We have been so focused on making models bigger that we forgot to make them cheaper. And “cheaper” here does not mean dollars. It means joules.
This is a hardware argument dressed up as an AI paper, and that is what makes it interesting.
Where it gets personal
The paper’s section on in-memory computing reads like a description of my day job.
I work on compute-in-memory accelerators — mixed-signal architectures where a digital input engine feeds an analog crossbar array programmed with neural network weights. The matrix-vector multiplication that dominates inference workloads does not happen in a processor. It happens in the physics of the device itself. Ohm’s law handles the multiply. Kirchhoff’s current law handles the accumulate. The computation is the current flowing through the array. No data movement. No memory wall. No fetch-decode-execute cycle burning energy to shuttle numbers between a processor and a memory bank.
This is not a future technology. It exists. It works. And it maps directly onto the paper’s thesis about where the next thousand-x comes from. When the paper talks about near-memory computing and eliminating data movement, they are describing the conservative version of what analog compute-in-memory already does. We do not put compute near the memory. We make the memory be the compute.
The efficiency numbers are real. A crossbar performing a VMM — vector-matrix multiply — consumes orders of magnitude less energy than the equivalent operation on a digital accelerator, because the physics does the work that transistors would otherwise have to clock through one cycle at a time. The trade-off is precision — analog is noisy, and you live with the noise or spend area and power cleaning it up. But for inference workloads where 4-bit or 8-bit precision is sufficient, the trade-off is not even close. Analog wins by factors that make the paper’s ten-x hardware target look modest.
So when thirty researchers argue that by 2035 the majority of AI inference will execute on edge devices with processing power approaching current cloud GPU levels while consuming a hundred to a thousand times less energy — that is not a prediction about the future. That is a description of the architecture sitting on my desk. The question is not whether the physics works. It is whether the ecosystem — the compilers, the toolchains, the frameworks, the software stack that I spend my days building — can make it accessible enough that people actually use it.
The physics was the easy part. Building the ecosystem around it — the CUDA-equivalent that makes developers reach for it without thinking — that is where the real moat gets dug.
The co-design thesis
The most important idea in the paper is not any single technology. It is the argument that hardware and software must be designed simultaneously.
This sounds obvious. It is not.
The way the industry works today is sequential. Someone designs a chip. Someone else writes a compiler for it. A third group writes a framework that targets the compiler. A fourth group trains a model using the framework. By the time the model meets the silicon, four layers of abstraction have accumulated, each one hiding inefficiencies that compound multiplicatively.
The paper argues for tearing that apart. Design the algorithm and the architecture together. Let the hardware team inform algorithmic choices. Let efficiency requirements drive architectural decisions. Build feedback loops that are weeks, not years.
This is not new in principle — Google did it with TPUs, achieving seven times the efficiency of GPUs by matching the hardware to the workload instead of the other way around. What is new is the scale of the ambition: doing it across the entire stack, for the entire field, in ten years.
The four bets
Reading between the lines, the paper makes four bets about where the next decade goes.
Bet one: sparsity is the free lunch we have been ignoring. Neural networks contain seventy to ninety percent sparse activations. Most of the computation in a forward pass is multiplying by zero. We have known this for years and mostly shrugged. The paper argues that exploiting sparsity — in hardware, in compilers, in algorithms — is the single largest efficiency lever available. Not the flashiest. The largest.
Bet two: the memory wall matters more than the compute wall. We have been obsessed with FLOPS. The real bottleneck is moving data. Memory access consumes ninety percent of execution time in many workloads. The paper proposes near-memory computing, 3D stacked memory, in-memory architectures, photonic interconnects — all aimed at the same problem: stop moving data to the compute, start moving compute to the data.
Bet three: quantization will get aggressive. Going from 32-bit to 8-bit is old news. The paper pushes toward 4-bit, 2-bit, even binary representations. The question is not whether you lose accuracy — you do — but whether the accuracy loss matters for the task. For most inference workloads, it does not. A model that is ninety-eight percent as accurate but runs on a hundredth of the power is not a compromise. It is a better product.
Bet four: the cloud is for training, the edge is for everything else. By 2035, the paper predicts, cloud AI will be relegated to training and research. Inference — the thing that actually touches users — will happen locally. This is not a prediction about technology. It is a prediction about economics. Edge inference at sub-watt power is simply cheaper than cloud inference plus network round-trips plus bandwidth costs plus latency penalties. The math wins eventually.
And it is already happening. Apple’s M5 Pro and M5 Max, released weeks ago, embed a Neural Accelerator into every GPU core — not as a sidecar, but as a first-class citizen of the architecture. The M5 Max delivers four times the peak GPU compute for AI over the previous generation. It runs large language models locally on 128GB of unified memory at 614GB/s bandwidth. No cloud round-trip. No API call. No latency penalty. A laptop, doing inference that required a data center rack two years ago.
This is not a research prototype. It is a product you can buy at the Apple Store. The edge is not a ten-year prediction. It is a March 2026 press release.
What has to break
The paper is honest about what stands in the way, which is refreshing for a vision document.
Materials science has to advance beyond silicon’s fundamental limits. Manufacturing has to scale specialized hardware without prohibitive costs. The entire ML software ecosystem — TensorFlow, PyTorch, everything — needs to be retooled for heterogeneous, low-resource targets. We need fundamentally new algorithms, not incremental improvements to existing ones. And we need standardization: hardware and software interfaces that enable interoperability instead of vendor lock-in.
That is not a to-do list. That is a decade of coordinated effort across materials science, device physics, chip design, compiler development, algorithm research, and systems engineering. The paper calls for public-private partnerships and coordinated initiatives, which is academic speak for “this is too big for any one company.”
The part they do not say
Here is what the paper implies but never states plainly: the current approach to AI scaling is a dead end.
Not because the models will stop getting better. They will keep getting better. But because the cost of making them better is growing faster than the value of the improvement. We are on an exponential curve of energy consumption for a logarithmic curve of capability gain. The lines cross eventually. Some would argue they already have.
The 1000x efficiency target is not about making AI cheaper for the people who can already afford it. It is about making AI possible for the people who cannot. Right now, frontier AI is accessible to maybe a dozen organizations on the planet. The paper envisions a world where comparable capabilities cost less than a million dollars in equivalent compute instead of a hundred million. That is not a technical improvement. That is a shift in who gets to participate.
Why this paper matters
Most vision papers are forgotten before the ink dries. This one might stick, for two reasons.
First, the authors are not dreamers. They are the people who build the systems they are writing about. When Christos Kozyrakis talks about memory architecture or Azalia Mirhoseini talks about chip design optimization, they are describing problems they have personally solved at scale. The credibility is earned.
Second, the paper’s central argument — that efficiency, not capability, is the binding constraint on AI’s future — is becoming harder to ignore with every new data center that breaks ground. The industry has been in a capability race. This paper is an argument for an efficiency race. And as anyone who has worked on edge hardware will tell you, efficiency races produce better engineering. Constraints tend to do that.
The hard part was never making AI smarter. The hard part is making it cheap enough that it matters.
That is the race worth running.
AI+HW 2035: Shaping the Next Decade is available on arXiv. If you build things at the intersection of hardware and AI, it is worth your afternoon.