Agentic AI,
Artificial Intelligence & Machine Learning,
Next-Generation Technologies & Secure Development
CIOs Build New Budget Models for Agents, Tokens and Failed Experiments
In the rapidly evolving landscape of technology and education, the integration of artificial intelligence (AI) presents unique challenges and opportunities for Chief Information Officers (CIOs). One such CIO, Zach Lewis of the University of Health Sciences and Pharmacy in St. Louis, confronts a common question when discussing AI expenditures with his board: “How much are we spending on artificial intelligence?” Yet, rather than providing a straightforward answer, Lewis responds with a clarifying inquiry that underscores the complexities of the situation: “Define AI. What do you want to know about specifically?” This dialogue illustrates the multifaceted nature of AI investments, as costs extend across various sectors and technologies within the institution.
AI’s integration into existing systems and software makes it challenging for institutions to delineate specific expenditures solely attributable to AI. For instance, while costs associated with standalone applications like ChatGPT and Microsoft Copilot are easily identifiable, identifying expenses related to AI functionalities embedded in pre-existing software becomes significantly more convoluted. Lewis points out that even fundamental tasks such as cleaning and classifying data generate costs that may not traditionally fall under AI budgets. He adds, “Does that count in your spend of AI? Because it’s a project for AI, but it’s not really AI.”
As organizations continue to adopt AI technologies, a trend emerges where costs proliferate across extensive areas, including cloud services, software contracts, data management programs, and even employee workloads. Before being able to demonstrate any return on investment (ROI), CIOs must first establish a clear understanding of what they’re actually spending on AI-related initiatives. JT Thykattil, a vice president and research director at Forrester Research, likens enterprise AI spending to an iceberg. The visible portion, which includes subscriptions for copilots, models, and applications, is evident. Yet below the surface lies an extensive mass of expenditure related to data preparation, system integration, training, and establishing governance and security measures.
The complex cost structure created by AI agents further complicates budget forecasting. Unlike traditional employee interactions, where a limited number of prompts are issued, an AI agent might execute hundreds or even thousands of calls during a single task. Consequently, this can significantly inflate token usage and, therefore, spending. In response to these challenges, Lewis and his IT teams have implemented data loss prevention measures aimed at identifying employee AI activities as well as independently created applications within the institution. “You don’t really know exactly what all their agents are doing,” Lewis notes, capturing the potential for unexpected costs that may impact AI budgeting.
To manage these sprawling expenditures, Lewis collaborates closely with the university’s Chief Financial Officer (CFO) to consolidate AI-related invoices spanning cybersecurity, IT, human resources, legal, and marketing sectors. This initiative aims to collate all these costs under a unified budgetary framework. “Let’s group them all together and try to roll that up under one blanket,” Lewis states, highlighting the pressing need for comprehensive financial oversight as vendors continue to interject AI capabilities into established products. As Lewis points out, “Almost every platform we have has some sort of AI agentic use now, or MCP server or something. We’ve seen costs increase on most of those.”
Amidst this evolving landscape, Lewis remains vigilant in discussions with vendors to ensure the preservation of fixed pricing structures, despite potential spikes in usage driven by more intensive applications of agentic features. Although the university has yet to receive unexpectedly high token bills, Lewis anticipates that more suppliers will introduce usage thresholds and associated overage fees.
In preparation for the upcoming budget cycle, Lewis anticipates a growth of approximately 20% in base licensing for several software products as AI features become more prevalent. He has also set aside a distinct pool of resources labeled the “failed experiments” budget. “It’s a play bucket,” Lewis describes. “We’re putting a set amount of money in there because we’re trying this, we’re exploring this. It’s not all going to work.” This proactive approach allows the university to experiment with AI initiatives without the burden of precise fiscal forecasting, while also rendering the costs of failure transparent to stakeholders.
Lewis has consistently briefed the board on AI initiatives at least once or twice annually for nearly two years, offering updates on various licensing scenarios, ongoing projects, and expected expenditures. The board has accepted the notion that certain experiments are necessary steps towards understanding where AI can generate real value. However, Lewis acknowledges that this tolerance for failure is not unlimited. “If I go the next couple of years and never have a return in any way that’s useful, they might be like, ‘All right, I think it’s maybe time to shut that down,’” he cautions.
As the University of Health Sciences and Pharmacy continues to evolve its AI capabilities, some applications have already progressed beyond the experimentation phase. Agents are now assisting departments such as records and accounts payable in querying contracts, determining termination dates, and assessing projected payment obligations. The university is also tapping into AI to analyze historical data seeking out potential student recruitment opportunities that traditionally went unnoticed.
Though Lewis’s early outcomes illustrate how CIOs can correlate AI projects with tangible business improvements, Thykattil warns that immediate returns might not be apparent in every case. “If you do a one-year ROI model, I think you’re doing yourself a disservice,” he argues, advocating for a multi-year perspective on such investments.
To effectively fund AI projects and garner the support of finance teams, Thykattil suggests that CIOs present a manageable number of pilot programs, defined funding requirements, and anticipated outcomes. He emphasizes the need for flexibility, as actual costs may fluctuate. Additionally, decision-makers should contemplate the opportunity cost of delaying AI initiatives while competitors advance their capabilities and employees cultivate essential AI skills. “The cost of business in the short term is increasing,” Thykattil concludes, but the potential long-term benefits that AI may offer could outweigh these immediate financial hurdles. For CIOs like Lewis, the immediate imperative is to ensure visibility into spending while adequately managing its growth in this dynamic landscape.