In an ever-evolving technology landscape, the rise of artificial intelligence (AI) is prompting enterprises to reassess data platforms that have undergone years of modernization. The insights shared by industry experts elucidate how the intersection of AI capabilities and traditional data architectures is reshaping enterprise strategies.

Modern cloud data warehouses, primarily designed for dashboards, financial reporting, and business intelligence, are now encountering new demands to provide support for AI models. These models require rapid data processing, enhanced governance practices, and a comprehensive understanding of enterprise data for effective operational decision-making. Consequently, CIOs are now faced with a crucial decision: should they extend the existing warehouses, pivot towards a data lakehouse, or construct a more specialized data architecture? The answer to this question hinges on the specific workloads an organization aims to automate and the degree of complexity it is capable of managing.

However, architecture represents just one piece of the puzzle. The successful implementation of enterprise AI also mandates robust semantic and governance foundations. These foundations enable AI systems to select the appropriate data while applying consistent definitions, ensuring that information is relevant to business processes. Bharat Bansal, a partner in Bain & Company’s technology and data practice, aptly points out, “The LLM, everybody can get. It’s your data that will distinguish one company’s AI strategy from another.”

Your Modern Platform Isn’t So Modern Anymore

The architectural challenges facing enterprises are surfacing due to the increasing variety of workloads being placed on a common data foundation. The emergence of agentic AI, which can retrieve information, make decisions, and act autonomously, underscores the necessity for high data quality. A data-quality issue that results in a faulty dashboard could lead to an autonomous agent making erroneous purchases or critical supply-chain decisions without human oversight. Thus, data quality and access control have rapidly evolved into vital operational safety necessities. Bain’s recent report on re-architecting the data platform for the AI era emphasizes this notion.

Bansal identifies two prevalent weaknesses in existing data environments: the separation between structured data, commonly found in warehouses, and the less-governed unstructured data stored elsewhere. Additionally, the lack of business context available to AI agents poses a significant concern. While many organizations may have data dictionaries and catalogs, agents require more than just definitions to make informed decisions. They need to comprehend the intricate relationships between customers, products, and interactions across various channels.

As Bansal notes, “The traditional semantic layers don’t provide that context to our agents,” suggesting that much of this critical knowledge still resides within the minds of employees.

Warehouse and Lakehouse Capabilities Are Converging

To prepare data for AI applications, Bain advocates for enterprises to extend their existing cloud data warehouse configuration rather than replace it outright. Modern platforms such as Snowflake, Google BigQuery, and Amazon Redshift have started incorporating open table formats, machine learning integrations, and vector-search capabilities, making them more versatile. For organizations primarily focused on business intelligence, reporting, and moderate generative AI applications, extending the existing warehouse can provide a quicker and less complex pathway forward, according to Bain’s analysis.

On the other hand, a fully operational lakehouse may be more suitable for organizations that demand substantial machine learning capabilities, real-time data access, or collaborative teams working from a common governed storage layer. Bain’s research indicates that highly specialized, best-of-breed architectures are likely appropriate for only 5% to 10% of organizations, due to the complexities of integration, governance, and financial management involved.

Bansal emphasizes the importance of aligning the data architecture with business needs, stating, “You don’t want the most sophisticated platform and architecture if it’s too complicated for the company’s needs or exceeds its ability to operate it.”

Noel Yuhanna, a vice president and principal analyst at Forrester, asserts that the lakehouse serves as a foundational element of a long-term AI-ready data architecture. He highlights its ability to accommodate SQL analytics, data science, machine learning, and AI within a unified platform, facilitated by open formats that promote data utilization without tying organizations to a single vendor. Nevertheless, he cautions against completely dismantling existing systems and encourages companies to evaluate whether their current platforms can meet evolving requirements.

AI and the Semantic Layer

CIOs must recognize that in data architecture, the semantic layer plays a pivotal role by sitting between the underlying data and the applications, analytics tools, and AI systems utilizing it. This layer establishes common definitions and supplies the necessary business context to interpret enterprise information accurately. For instance, while an employee may grasp varied meanings of “sales” among different teams, an AI agent needs clearer parameters to differentiate between net sales that account for cancellations and gross sales figures that are often used for commission calculations.

Josh Fecteau, the chief data and analytics officer at Teradata, elaborates on why ROI from AI initiatives is frequently more apparent at the individual level than on a broad enterprise scale. Individual employees using chatbots can provide context and clarify misunderstandings, while an autonomous enterprise agent must independently discern and utilize the correct data. He insists on the necessity of a well-defined data framework, asserting, “You have to have a fully described set of data and truth out of the gate.”

Importantly, this doesn’t imply that all data must be physically consolidated; instead, organizations need consistent metadata, definitions, and governance across their entire data landscape. Fecteau warns, “If you have too many sources of truth, you have no source of truth.”

Build the Foundation Through Business Use Cases

Developing a catalog, lakehouse, or semantic platform will not resolve issues concerning data ownership. Although CIOs can equip organizations with technology to implement business processes, the resolution of ownership disputes ultimately rests with departmental leaders. They must delineate who owns the data, as well as who bears responsibility for maintaining the data and evolving definitions as workflows change.

Bansal emphasizes the necessity for clear ownership to secure investment in a data platform, stating, “You can have the best platform, but nobody will invest the resources behind it unless ownership is clear.” Furthermore, he notes that many organizations are failing to invest sufficiently in data engineering and architecture. While it’s not uncommon for businesses to hire machine learning engineers, these talented professionals often end up bogged down with data cleaning and pipeline construction, which can limit the return on their expensive expertise.

To enhance the value of AI initiatives, he recommends tying investments in data platforms to specific use cases, rather than relying on a broad request for funding from a CFO to improve data quality in general terms. For organizations aiming to implement AI for customer support or supply-chain management, identifying the necessary data and ensuring it is AI-ready becomes paramount. “The value is in the customer support business case,” Bansal asserts. Thus, the semantic layer and data pipelines transform into essential components of the investment strategy rather than vague platform enhancement projects.

Teradata’s approach illustrates this perspective effectively. The company crafted AI applications around complex contract information, enabling data sharing across agentic applications and constructing a foundational use case. Fecteau explains that this initiative yielded approximately 100,000 hours of operational capacity savings. The knowledge generated from this project was then repurposed to develop additional applications, such as a sales intelligence agent that combines contract and customer data. This cumulative return on investment deepens as multiple divisions and applications draw from the insights gained through foundational efforts.

Ultimately, identifying scenarios where fully described data sets can deliver immediate value creates compelling proof points for justifying additional investments. As Fecteau succinctly states, “It’s not just a proof of concept. It’s an actual use case that’s running in production, where you can say, ‘Well, here’s the proof.’”