AI systems evolve continuously, even when the foundational model remains unchanged. Various elements such as retrieval indices can update overnight, new tools may be integrated into an agent’s operational capacity, and an assessment that received approval on one day may quickly become irrelevant by the next. This dynamic nature of AI contradicts the underlying assumption of a compliance-as-review approach, which presumes static conditions between review cycles. Many organizations have yet to adapt their governance strategies for AI, frequently relying on traditional software management models that often involve a sequence of building, shipping, and then seeking legal approval—a practice that can obscure the continuous developments and modifications in AI systems.
In the course of researching the approaches different countries take toward AI governance for her upcoming book on the AI ecosystem in China, the author uncovered insights that challenged her initial beliefs. Unlike the typical methodologies observed in many Western organizations, Chinese AI companies do not view governance as a mere post-development checkpoint. Instead, these companies consider governance to be an integral part of the release infrastructure itself. Compliance is not a hurdle that needs to be cleared after a model is deemed functional; it is embedded within the deployment pipeline. In this setup, no product launch occurs without passing specific governance checkpoints, effectively merging governance with product development.
During a review of an AI deployment, the author witnessed a scenario that exemplified these differences in approach. The product team had all the typical components prepared for a launch meeting, such as performance metrics, customer use cases, latency numbers, and a definitive release date. However, several critical elements were conspicuously absent from their preparations. There was no current, pipeline-generated record of the retrieval index being utilized to power the model, nor was there anyone assigned to oversee the output-monitoring thresholds. Additionally, the outcomes of model evaluations lacked any connection to enforceable release gates. While the team was not actively neglecting governance, it became evident that no suitable framework existed within the actual release process for governance to effectively reside or be executed.
This situation is becoming increasingly prevalent across organizations. When governance operates outside the engineering workflow, it often finds itself at odds with delivery timelines. In this conflict, the urgency of meeting deadlines almost always prevails. The NIST AI Risk Management Framework outlines four core functions for managing AI risks: governing, mapping, measuring, and managing. However, it falls short of specifying where these functions should be situated within a release process, leaving the intricate architectural decisions to fall upon security organizations. As a result, many companies revert to familiar patterns, relying on a periodic review cycle borrowed from traditional IT compliance frameworks designed for systems that remain static during audits.
Such prevailing methodologies expose organizations to significant risks. As AI technologies experience rapid iterations, the governance mechanisms that lag behind become outdated quickly. This disconnect not only undermines the efficacy of compliance efforts but also jeopardizes the integrity and safety of AI systems. Without an effective governance structure integrated into the release pipeline, organizations may inadvertently overlook essential compliance measures, which can lead to unintended consequences—ranging from ethical violations to operational failures.
Organizations must recognize that AI is not static and that its governance must evolve accordingly. The dynamic landscape of AI development necessitates a more integrated approach to compliance, where checkpoints and evaluation criteria are interwoven into every stage of the deployment process. By rethinking how governance is structured and implemented, organizations can ensure that their AI systems not only meet regulatory standards but are also aligned with ethical considerations and operational excellence.
Ultimately, the challenge lies in adapting governance frameworks to better reflect the realities of AI development. The lessons learned from organizations adept at embedding compliance into their workflows offer valuable insights for others grappling with the complexities of governing AI. It’s essential that organizations proactively rethink their approaches to AI governance, allowing for a governance structure that is as agile and evolving as the technologies they seek to regulate.