A growing concern within the realms of technology and cybersecurity is the effective management and visibility of AI-driven tools, often referred to as “harnesses.” According to industry expert Santos, teams across organizations are increasingly conceptualizing their AI assets in terms of apps, services, pipelines, or bots, leading to a fragmented understanding of how these elements function and relate to one another. Santos notes that harnesses have a tendency to become submerged in code repositories, software-as-a-service (SaaS) products, and vendor configuration screens, rather than being recognized as distinct assets within security inventories.
The terminology associated with these AI components has also become a point of contention. Santos points out that disparate teams within the same organization may refer to similar AI entities using various terms. For example, one team might label something as an “agent,” while another might term it a “copilot,” “workflow assistant,” or “plugin-based automation.” Despite these differing names, Santos emphasizes that intrinsically, all these designations describe what are essentially harnesses.
To address these complexities, Santos urges organizations to establish a comprehensive live inventory of every production agent in use. This inventory should also identify the harness associated with each agent and map out every tool and resource that the agent can access. The underlying objective is to streamline and minimize permissions, ensuring that agents can only access what is absolutely necessary for their functions. Santos asserts that organizations should not wait for an ideal state of complete visibility. He estimates that achieving 60% to 70% visibility can be accomplished relatively swiftly by focusing initially on production systems, reserving more complicated proprietary systems and shadow AI for future phases.
Furthermore, Santos highlights a second critical challenge: controlling the trust relationships that harnesses establish. Unlike traditional software, agents don’t function autonomously; they often process instructions and content from a plethora of tools, plugins, skills, master control (MCP) servers, websites, and other systems. In doing so, they typically hold credentials and permissions that enable them to operate on behalf of users, making it easier for malicious actors to exploit these trust relationships rather than directly attempting to target the models themselves. Bargury aptly illustrates this concern by explaining that when users share their laptops with agents, they inadvertently share their entire digital identity, including files, secrets, and other sensitive information.
For organizations that do not have a dedicated budget for AI security, Bargury recommends utilizing open-source containment tooling to run agents, although he cautions that this approach is merely a stopgap solution. He warns that this measure, while providing a layer of security, is not a comprehensive fix.
The breadth of what constitutes potential supply chain vulnerabilities further complicates matters. Bargury contrasts the simplified supply chain for traditional software, which generally involves a mere ten to fifteen package registries, with the vast and intricate supply chain for agents. This complexity encompasses any content, image, or website existing on the internet, as well as CRM objects and any number of other online resources. The exponentially larger attack surface makes effective management much more daunting.
Another critical issue arises from placing undue trust in vendor claims regarding security efficacy. Bargury raises concerns about vendors who assert that their systems block 99% of prompt injections. He points out that these claims may be referencing benchmarks that bear little resemblance to real-world operational environments, thus failing to provide organizations with an accurate assessment of safety.
Additionally, research conducted by Lasso underscores the significance of this issue. Their findings indicate that when the model, prompt, and tools remain constant but the harness changes, the security outcomes can vary dramatically. This revelation suggests that organizations may be asking the wrong questions when evaluating agents. Rather than simply trying to determine which model is the safest, they should consider the interplay between model, harness, tools, permissions, and external inputs to ascertain what will remain secure within their specific operational context.
Meged further encapsulates the principle that organizations should focus on the defaults of their systems rather than relying solely on vendor documentation. He observed that oftentimes, the product literature touts safety without adequately addressing the underlying vulnerabilities inherent in its configurations. His experience with three different vendors exemplifies the necessity for organizations to be vigilant in reviewing the defaults before implementing potentially risky automated systems.