AI Labs' Single-Minded Focus on Transformers Risk Missing True AGI, Expert Warns

By

Breaking: AI Industry's Transformer Bet Faces Growing Skepticism

The global AI industry is pouring tens of billions of dollars into a single approach—pre-trained transformer models—in the pursuit of human-level general intelligence. But a leading expert warns this one-track strategy may be a costly mistake.

AI Labs' Single-Minded Focus on Transformers Risk Missing True AGI, Expert Warns
Source: www.fastcompany.com

Ben Goertzel, the researcher who coined the term “AGI” in 2005, argues that the obsession with scaling transformer models is diverting resources from more promising paths. “The commercial AI industry is just betting everything on copying GPT in various permutations, which in my view is a waste of resources because all these LLMs are kind of doing about the same thing,” he said.

Major labs like OpenAI, Google DeepMind, and Microsoft are doubling down on transformers, relying on backpropagation and ever-larger datasets. But Goertzel warns that this concentration poses existential risks to the AGI timeline.

Quotes from the Expert

“When something works, everyone wants to double and triple down on what worked,” Goertzel told Fast Company. He emphasizes that transformers require billions in compute—both for training and ongoing operation—and that the returns are diminishing.

While current labs report incremental gains from adding more compute, those gains come at skyrocketing expense. “The financial stakes are so high that labs have little room to invest seriously in fundamentally different approaches,” Goertzel said.

He also pointed out a core limitation: “Transformers cannot continually learn from new experiences and update their internal parameters in real time the way humans do. Instead, they revert to baseline parameters with each new interaction.”

Background: The Transformer Paradigm

Transformer models, introduced in 2017, have become the dominant architecture for AI. They rely on backpropagation—a standard algorithm for training deep neural networks—and massive pre-training on text, images, or code.

Companies like OpenAI have scaled these models to unprecedented sizes, but the approach requires billions of dollars in compute and data. The risk is that without novel algorithms, scaling alone won't cross the chasm to AGI.

Inside the Labs: Alternative Approaches

Despite the industry's focus, some teams are exploring other architectures. Goertzel notes that researchers at Google DeepMind, Microsoft, and Ilya Sutskever’s Safe Superintelligence are investigating alternatives that enable continual learning.

“DeepMind has incredible diversity within their AI team” and a “deep bench” of experience with alternate paradigms, Goertzel said. However, these efforts remain overshadowed by the transformer juggernaut.

What This Means for the AI Industry

The current landscape means that massive compute resources are largely devoted to refining existing methods rather than pursuing fundamentally different architectures. This could delay the arrival of true AGI—if it ever arrives via current paths.

Goertzel remains optimistic that AGI could emerge within the next few years, but only if labs diversify. “It will likely require moving beyond simply scaling current LLMs,” he said.

Investors and tech leaders face a critical decision: continue pouring billions into transformers, or back a broader portfolio of approaches. The next few years may determine whether AGI becomes a reality—or a mirage.

—Reporting contributed by AI Decoded staff

Tags:

Related Articles

Recommended

Discover More

Building Smarter AI Systems: A Practical Guide to the Probabilistic Paradigm ShiftInvincible Vs Data Mine Reveals Potential Roster Expansion: Exclusive Characters LeakedNew Facebook Groups Search Overhauls Discovery, Consumption, and Validation of Community KnowledgeApple Discontinues Entry-Level Mac Mini, Raises Starting Price to $7997 Critical Lessons from GitHub's Git Push RCE Incident