Token Spend Is New Cost: Enterprises Race To Manage AI Consumption

Enterprises are deploying more AI agents than ever before. Agents, however, need more computing power. Subsequently, it’s pushing AI token consumption – also emerging as a new enterprise cost. 

Goldman Sachs says AI agents will lead to 24x AI token consumption in the next four years. 

“…by 2030 agentic AI will multiply token consumption 12 times on the consumer side, so things like online shopping, cell phone takeovers, and similar functions. Combined with adoption by enterprises, that is 120 quadrillion tokens processed per month,” according to a Goldman Sachs post earlier this year. 

According to the world’s top model providers, OpenAI and Anthropic, costs of tokens are going down, and slashing prices. But is the token cost reduction in ratio with the growing consumption of AI? Apparently not. 

Even as average AI token prices have declined by nearly 19%, overall token consumption is estimated to grow exponentially by 78% in the next couple of years, according to an Accenture research. Moreover, AI budgets for enterprises are going to skyrocket by up to 320% despite lower unit costs. 

The above-mentioned Accenture research says that four out of five executives from its survey credited AI for measurable outcomes. 

“But fewer than one in five can connect value to token spend. Most struggle to explain such a spend in financial terms that a CFO can act on, the CEO can quantify and a board can evaluate,” it adds. 

“The trajectory is the real concern: companies expect token consumption volume to grow 78% over the next 24 months. Even after accounting for a predicted 19% price decline, executives expect that, over the same period, aggregate token spend will approach $3.6 billion, with no optimization applied. Behind that aggregate, the range is vast: a global technology company spending north of $20 million annually, a European insurer with an AI inference budget of $150-200 million, a government contractor with a global AI spend ceiling of $250 million,” the report continued. 

The aggressive push has nothing to do with the AI prices. Even if it falls by another 25%, according to the Accenture report, only one in 20 said they would prefer saving. 

Earlier this year, a new buzzword emerged – tokenmaxxing. IBM explains it as an “informal term for the organizational practice of incentivizing employees to maximize their AI token usage, typically as a means to encourage experimentation with AI tools and automation of everyday tasks.” 

This led to several tech giants to crackdown on the so-called tokenmaxxing as it began setting a new – and probably wrong – precedent wherein measurement benchmark tilted towards input volume rather than actual output or quality. 

Just recently, Amazon in a memo, circulated internally in September and later made public, described tokenmaxxing as a “deeply flawed” metric. 

“We’re just seeing the beginning of AI’s effect on software engineering,” the company wrote in the memo. “While we have a vision for what the future looks like, there’s a lot that’s still emerging. We are certain that whatever future best practices are, AI will be at the center of software engineering practice.”

Not that the West was not ahead in the AI race already, it also has found another advantage over their counterparts in emerging markets. It’s the language. 

According to a new Clear Languages study titled ‘The Tokenizer Tax’, AI models consume more tokens for the same sentence in certain languages over others. 

“For most LLMs, the disparity in tokenization efficiency across languages works like a regressive tax: the less digital presence a language has, the higher the rate its speakers pay. This tax rate varies enormously by model,” the report observes. 

This means Indian enterprises especially using vernacular languages must think carefully about how they use languages in their AI system. 

Ashok Kadsur, Co-founder of Melento (formerly SignDesk), tells Entrepreneur India that whilst not all processing steps need to happen in the customers native language, businesses can take advantage of language aware representations, structured representations and multilingual specific models to reduce the token overhead. 

“The key thing to remember is that language should not be a cost to the business. Customers should have the option to interact in their language of choice (be it Kannada, Telugu or Hindi) without affecting the quality of service they receive. The key thing to remember is that India’s AI opportunity relies on being able to provide a viable solution for multilingual intelligence.”

The onus is clearly on the AI companies to democratise the access. And it must not be restricted to just providing free credits and open APIs. 

According to Kadsur, it’s also about providing low-cost inference, regional hosting, efficient open weight models and enterprise security with no high minimum spend for smaller businesses. Although there is now more access to intelligence, there are still challenges in making it economically viable at scale. AI providers could look to provide infrastructure packages aimed at startups with predictable costs and affordable enterprise-ready options.

“This will help reduce the costs of trying new ideas and adopting AI commercially. Real democratisation means that a startup can innovate based on how good their idea is rather 

than how much they can afford to spend on expensive AI infrastructure,” he continued.

And then there’s this symmetry in the ecosystem, mainly tilted towards the West. Earlier this year, Anthropic had developed a cyber-security focused model Mythos but disabled its access for its customers. 

Dhanda, Founder & CEO of Utho Cloud, tells Entrepreneur India that owning your data is not the same as owning your capability. 

“We can keep every byte inside India and still not control whether the technology our economy runs on works tomorrow morning. Think about what now depends on foreign AI in this country. Our banks. Our hospitals. Our startups burning runway. Our government departments. Our defence and research. All of it increasingly built on a handful of models owned by companies in another country, governed by a government that is not ours, that can withdraw access on a Friday evening for reasons it does not even have to explain to us,” he added.

He also noted that India has the talent, the engineers, the data centres and the demand to build a real sovereign stack. 

“Compute we control. A country that does not own its AI does not own its future. It rents it. And rent can be cancelled. So the question for all of us is simple. If the AI our country depends on was switched off tomorrow by a government that is not ours, what exactly is our plan? Dhanda added.

That said, experts believe that AI architecture will move towards more of a specialist team to ensure judicious consumption of AI tokens and that it does not turn into another large burden.

It’s unlikely that an enterprise expects senior resources to be deployed towards mundane tasks. One may see more adoption of smaller language modes that are used for repetitive, specialist and predictable tasks and frontier models can be used for more complex reasoning. 

“Prompt caching and intelligent routing improves this by removing any repetitive calculations. This means businesses can achieve a good balance between the intelligence, accuracy, latency and cost. The next generation of successful AI will be based on how intelligently they are using a mixture of models and automation,” Kadsur concluded.