Managing AI’s energy appetite: The role of small models

The transformative potential of artificial intelligence cannot be denied, but behind the scenes there is a disturbing truth – the training and deployment of powerful models, especially the large language models (LLMs), require enormous amounts of energy. With the energy restrictions that countries are beginning to impose on data centers and against the background of the huge increase in workloads of artificial intelligence, programmers, software developers and businesses are faced with a new reality, in which we are tasked with the task of innovating and developing in a world where energy is limited, and abandoning the old paradigm that assumed unlimited resource availability.

The latest regulatory moves in the US signal an irreversible shift towards a reality where energy efficiency is at the heart of AI infrastructure planning and AI deployment strategy. After several years in which countries such as Singapore and EU countries imposed similar restrictions, US states, including Texas, Oregon, Delaware and Illinois, are now introducing regulations designed to reduce the load of data centers on local energy networks. For AI developers, this means adopting alternative approaches that prioritize energy efficiency and smart computing infrastructure.

Fortunately, innovation in artificial intelligence is proving that effectiveness does not depend on size alone. While large language models have attracted the most attention, more and more AI development teams are turning to small language models (SLMs) as a focused, efficient and resource-efficient alternative that can be more suitable for enterprise use. These models often include less than 10 billion parameters, compared to hundreds of billions or even trillions in LLMs, and they are viable alternatives under conditions of computing limitations, fast processing and low latency. In fact, even OpenAI’s GPT-5 incorporates dynamic switching between small and large models depending on the complexity of the question, relying on small models for simple tasks and larger models for complex tasks.

Organizations have tended to underestimate SLMs in the past, due to fear of inferior performance at the organizational level, difficulty to expand or lack of extensive knowledge due to narrower training data. Many feared that these models would prove to be lacking when it comes to complex analysis, multilingual capabilities and effective dealing with nuance and ambiguity. However, many of the misconceptions about SLMs have been debunked in the past year and a half.

Underlying the move to small language models is the growing recognition that data quality, not quantity, is the key to good performance. Instead of huge, raw and unfiltered databases, some of which are also duplicated or irrelevant, the small models are based on so-called ‘data efficiency’ principles, where data sets are carefully curated, with accuracy and relevance in mind, which improves accuracy in targeted tasks and reduces energy waste.

In fact, the focused approach of developing small, dedicated language models, trained on carefully curated high-quality datasets, has significantly improved accuracy on specific tasks, while reducing the inefficiencies of large models. The small models excel in their ability to undergo fine-tuning and can be updated more frequently, which makes them flexible and ideal for dynamic environments. In addition, SLMs appeal to organizations due to their ease of deployment and management, and due to their suitability for edge environments, such as smart factories, where energy efficiency is a critical factor.

Many examples of successful use of small language models emerge today in a variety of fields. Retailers are using leaner AI models to power chatbots for customer support, while smartphones and wearables equipped with SLMs enable real-time translation. In the field of medicine, the models are adopted to support doctors who work under time and resource constraints, through a rapid analysis of patient symptoms, laboratory results and diverse medical information, which enables the production of possible diagnoses and the proposal of therapeutic measures. Beyond that, not only can SLMs be adapted to specific sectoral applications, based on in-depth domain-specific knowledge, they also stand out with faster analysis times, lower latency and reduced hardware requirements, features that make them efficient, accessible and sustainable tools for a variety of industries.

Alongside the rise in popularity of small language models, Open-Weights models are also establishing themselves as an energetically efficient alternative to large language models. Models based on the Blend of Experts (MoE) architecture attract developers who are aware of the importance of energy efficiency, because they run only a small part of their parameters at each calculation step, thus dramatically reducing the computing resources and energy required for each task.

So for example, a model with a hundred billion parameters may only need five billion at a time. And because open-weight models can be deployed locally and tuned to specific needs, they also lighten the energy load of cloud infrastructures, and are therefore particularly suitable for edge environments and enterprise applications where energy efficiency is critical. A recent example is the launch of gpt-oss-20b, an open-weight model that can run on standard user hardware with only 16GB of memory. The lessons from the last two years show that SLMs can be just as effective as LLMs, cheaper to operate, and succeed in tasks that do not require a particularly broad knowledge base.

To survive the next phase of AI development, organizations will have to oblige AI teams to place energy efficiency as a fundamental principle and consider energy implications at every stage of the AI ​​lifecycle, from data management, through model architecture design, to actual deployment. By applying holistic thinking based on a complete life cycle of IT challenges, developers will be able to refine their work practices and reignite creativity in the next wave of AI development.