Our CTO, Suhas Pai, was recently invited to the Canaccord Genuity AI Forum to share his thoughts on the changing AI landscape. The discussion focused on improving the performance of large language models, intensifying competition in AI, and what to expect from AI models in the future.

Suhas is a well-known applied AI, natural language processing and machine learning researcher. He is the author of the popular textbook, Designing Large Language Model Applications, published by O'Reilly Media.

‍

‍

Here are the key points Suhas covered in the discussion:

1. Latency and Performance: Balancing User Experience

A small increase in latency—how long it takes for a model to respond—can significantly boost performance. Users typically prefer immediate answers, so user interfaces need to encourage a brief wait if it means higher-quality results. Many current reasoning models overthink problems, producing complex answers that often miss the mark. This “overshooting” leads to wasted resources and unnecessary delays. Thought budgeting—allocating only the reasoning power needed for a task—helps models remain focused and deliver fast, accurate answers without extra complexity.

2. From Generation to Decision-Making

The next step for AI involves decision-making. Models should recognize when they lack enough information and then seek it out. This capability is crucial for advanced applications like scientific discovery, where AI must go beyond generating text to identify gaps and find missing pieces.

3. Data Efficiency and Smaller Models

Current models remain sample inefficient, requiring large datasets to learn new tasks. With better data mixtures—combining varied sources thoughtfully—smaller models can achieve results once reserved for larger systems. This shift makes AI more agile and cost-effective, particularly for specialized tasks with narrow goals.

4. Sovereign AI and Open Source

Sovereign AI—where countries and companies control their own AI infrastructure—has become critical. Many organizations depend heavily on cloud services like AWS, creating a single point of failure if these services are interrupted. In-house models ensure stability and independence. Open source models provide flexibility, creative freedom, and control over intellectual property. In contrast, closed source APIs limit customization and force reliance on opaque systems. Open source lowers barriers for experimentation and fuels genuine innovation.

5. The Changing Global AI Landscape

The global AI ecosystem has shifted dramatically. In 2023, eight of the ten top-performing models were American. Today, five are American, four Chinese, and one Canadian. Companies like DeepSeek have emerged as strong competitors, proving that innovation is no longer confined to a single region. This competitive landscape will likely continue to narrow the gap between closed and open source models, driving new breakthroughs from unexpected players.

6. Adapting to User Expectations

Early AI tools launched without clear user expectations. Now, systems like ChatGPT shape what people expect from AI. Even when a tool isn’t designed as a chatbot, users still anticipate conversational, chatbot-like interactions. Managing these expectations has become as important as developing the underlying technology.

7. Specialized Models for Specialized Tasks

Multiple models, each specialized for different tasks, are necessary. The largest, most advanced model isn’t always the best fit. For example, generating a product description doesn’t require the same capabilities as analyzing a dataset. Smaller, fine-tuned models often deliver better results at lower cost. Matching the right model to the task avoids unnecessary complexity and makes AI solutions more practical and sustainable.

8. The Cost of AI and New Frontiers

Using large language models for every task isn’t practical due to high costs and often limited context lengths—how much information they can process at once. As these limitations ease, with longer context windows and falling compute costs, many new applications will become viable. Soon, models will be small enough to run on individual laptops, giving developers and researchers more freedom and reducing dependence on cloud services.

9. Mixture of Experts and Infrastructure Innovation

The Mixture of Experts architecture once seemed promising, allowing models to select different “experts” for different tasks. However, early versions were inefficient and hard to scale. Recently, DeepSeek has improved both the architecture and supporting infrastructure, laying the groundwork for more scalable and cost-effective models. These advances could enable larger and more capable models without excessive computational overhead.

The Road Ahead

Competitive pressure is driving rapid innovation. OpenAI plans to open source a model soon in response to competition from players like DeepSeek. With lower costs and stronger open source communities, the future of AI looks bright and full of opportunity.

‍