For a while now, the buzz around Artificial Intelligence has often focused on the massive, all-knowing models like GPT-4 or Claude. These “monolithic” models are incredibly powerful, capable of understanding and generating human-like text, code, and more. However, a new wave of innovation is shifting the focus. Instead of relying on one giant brain, the future of AI looks more like a highly skilled team, where different specialized AI models work together to achieve complex goals. This shift, highlighted by developments like Sakana AI’s approach to combining models at inference time, marks a significant evolution in how we build and deploy AI.
Imagine trying to build a house. You could try to be a master builder, plumber, electrician, and roofer all at once, but you'd likely be slow and not very good at any single task. A more efficient approach is to have a team of specialists: a carpenter for the framing, a plumber for the pipes, an electrician for the wiring, and so on. Each specialist knows their job deeply and works with the others to create a functional whole.
This is precisely the analogy for the evolving AI landscape. Traditionally, AI development often aimed to create one massive model that could handle a wide range of tasks. While impressive, these models can be computationally expensive, difficult to update, and may not perform optimally across every single specialized task. Sakana AI’s vision, as described by The Sequence, embraces this “team” approach by seamlessly combining different AI models at the moment they are needed (inference time). This means an AI system can dynamically call upon the best-suited “expert” model for a specific part of a problem, much like assembling a specialized crew for different stages of construction.
One of the key architectural innovations enabling this shift is the Mixture-of-Experts (MoE). Think of an MoE model as a committee of expert AIs, each trained for a slightly different skill. When a new piece of information comes in, a smart “router” decides which expert (or combination of experts) is best equipped to handle it. For instance, in a large language model, one expert might be great at creative writing, another at factual retrieval, and a third at coding. The MoE architecture activates only the relevant experts for a given query, making the process more efficient and often more powerful.
Companies like Mistral AI have gained significant attention with their MoE models, such as the Mixtral 8x7B. This model, for example, uses eight “expert” networks, but only two are activated for any given token (a basic unit of text). This allows it to perform at a level comparable to much larger, denser models while being significantly faster and more efficient. This is a direct embodiment of Sakana's blueprint: leveraging multiple, specialized components that work in concert. As explained in resources discussing MoE, this approach is a fundamental step towards building more versatile and performant AI systems than a single, monolithic architecture can easily achieve.
For more on this, explore: Mixture-of-Experts Explained.
The move towards “model teams” is not just about technical architecture; it’s also about the behavior and capabilities of AI systems. This leads us to the concept of Agentic AI. An AI agent is an autonomous system that can perceive its environment, make decisions, and take actions to achieve specific goals. If monolithic models are like powerful calculators, agentic AI systems are more like intelligent assistants or even autonomous workers.
A collective of specialized AI models is a natural fit for building sophisticated AI agents. Imagine an AI agent tasked with planning a complex trip. It might employ a language model to understand your requests, a mapping AI to find routes, a booking AI to secure flights and hotels, and a calendar AI to manage your schedule. Each specialized model acts as a distinct "agent" within the larger system, collaborating to fulfill the overall objective. This makes AI systems more proactive, capable of handling multi-step processes, and adaptable to changing circumstances. The ability for these agents to communicate and coordinate, much like Sakana's combined models, is what will unlock truly advanced AI applications.
Learn more about this exciting field: Agentic AI: The Next Frontier.
The trend towards combining AI models reflects a broader movement in technology towards modularity. In software development, we’ve seen the shift from large, complex “monolith” applications to smaller, independent “microservices.” This modular approach offers significant advantages: easier to update, scale, and maintain. If one part fails, the whole system doesn't necessarily crash, and you can improve or replace individual components without rebuilding everything.
AI is following a similar path. Breaking down a massive AI model into smaller, specialized modules – for example, a module for image recognition, another for natural language understanding, and a third for sentiment analysis – provides immense flexibility. This “modular AI” approach means systems can be customized more easily. Businesses can assemble AI solutions by picking and choosing the best modules for their specific needs, rather than trying to find a one-size-fits-all monolithic model. This also allows for quicker iteration and improvement, as individual modules can be retrained or replaced with newer, better versions without disrupting the entire AI system.
The pursuit of “composable AI systems” – where different AI capabilities can be easily combined like building blocks – is a direct outcome of this modular trend. It’s about creating AI systems that are as adaptable and evolvable as the needs of the businesses and users they serve.
The idea of specialized AI components working together is also influencing how AI is made accessible to everyone. Platforms like OpenAI's GPT Store are a fascinating example of this at the application level. Here, users can create and share custom versions of large language models, tailored for specific tasks or knowledge domains. Each custom GPT is a specialized AI, an “expert” in its own right.
The GPT Store fosters an ecosystem where these specialized AIs can be discovered, utilized, and even combined by users. For instance, a small business owner might use one custom GPT to draft marketing copy, another to analyze customer feedback, and a third to generate social media content. The platform effectively creates a marketplace of AI expertise, mirroring Sakana’s technical vision of combining models, but in a way that empowers end-users to build their own AI workflows. This "democratization" means that powerful AI capabilities are no longer solely the domain of large tech companies but are becoming accessible tools for individuals and smaller organizations.
See how this is unfolding: OpenAI's GPT Store.
The move from monolithic AI models to dynamic, collaborative “AI teams” has profound implications:
For businesses, this evolution translates into tangible benefits:
For society, this means AI can become more integrated into our daily lives in more specialized and helpful ways. Imagine an AI assistant that seamlessly switches between understanding your health data (using a health-specific AI), managing your appointments (using a scheduling AI), and providing personalized nutrition advice (using a dietary AI) – all without you noticing the distinct components at work.
As this trend unfolds, here’s how businesses and individuals can prepare:
The journey from single, powerful AI models to collaborative, intelligent teams is well underway. This shift promises a future where AI is not just a tool, but a versatile, adaptable, and integrated partner in solving increasingly complex challenges.