In the realm of cyber/information security, the search for trusted advisors is ongoing. On platforms like LinkedIn, numerous consultants market themselves as trusted advisors to Chief Information Security Officers (CISOs) and their teams. The implication is clear: nobody wants to engage an untrusted advisor. However, when considering modern AI-powered chatbots and GenAI applications, it becomes evident that these technologies essentially function as untrusted advisors, despite the advancements in models like RAG and fine-tuning.

The effectiveness and minimized risks of using untrusted security advisors can be seen in various use cases, particularly in the realm of security operations centers (SOCs). It is crucial to note that the output of AI-powered applications should always be reviewed by a human with relevant domain knowledge. This oversight is essential to identify patterns, anti-patterns, and dependencies for successful utilization of untrusted advisors in security operations.

Tasks involving ideation, brainstorming, and refining ideas are well-suited for leveraging the capabilities of untrusted advisors. By generating ideas for security architectures, controls, and approaches, these AI tools can accelerate the creativity process and shorten the time to value. Moreover, scenarios such as Devil’s Advocate use cases, “what-if” scenarios, security testing, and report drafting can benefit from the input of untrusted advisors, provided that human experts review and validate the generated content.

While there are clear benefits to utilizing untrusted advisors for certain functions, there are also notable risks and limitations. Direct deployment of controls, automated reconfiguration without human review, and sharing detailed knowledge of the environment with untrusted advisors are all high-risk activities that should be avoided. However, by maintaining a clear separation between untrusted outputs and critical systems, organizations can bridge the trust gap and leverage the benefits of AI technology.

Ultimately, the responsible use of AI-powered untrusted advisors in cybersecurity requires a balance between human expertise and AI assistance. Ideation, testing, and red teaming are valuable applications for untrusted advisors, while direct control, access to sensitive data, and unsupervised deployment are areas where caution is warranted. Human oversight and validation of AI outputs remain crucial to ensuring the safe integration of these technologies with critical systems.

In conclusion, leveraging AI-powered untrusted advisors can be a valuable asset for security teams when used responsibly. By focusing on appropriate use cases, maintaining human oversight, and striking the right balance between human expertise and AI assistance, organizations can harness the benefits of AI technology while mitigating potential risks. The evolution of AI technology continues to present exciting opportunities for improving cybersecurity practices, but a cautious approach is necessary to ensure the safe and effective integration of these tools in security operations.

Link na izvor