In the evolving landscape of cybersecurity, AI, generative AI, and AI agents are becoming increasingly prominent tools used by both cyber criminals and defenders. Anna Collard, the Senior Vice President of content strategy and evangelist at KnowBe4 Africa, shed light on this trend during a recent webinar on AI in cyber risk.
According to Collard, a poll conducted during the webinar revealed that 95% of participants expressed concern about the impact of AI on cybercrime. This sentiment reflects a global trend where cybercriminals are leveraging AI to scale and automate their malicious activities. Tools like generative AI are now readily available to criminals, enabling them to enhance existing tactics such as social engineering. Egress, a company recently acquired by KnowBe4, conducted research showing that a majority of phishing toolkits now incorporate AI features, and 82% of criminal toolkits include deepfake capabilities.
Moreover, Collard highlighted the proliferation of cognitive and narrative attacks as well as disinformation campaigns that manipulate public perception and erode trust in democratic institutions. She pointed out that targeted disinformation campaigns on social media in Africa have quadrupled since 2022.
While the use of AI by malicious actors poses significant risks, Collard also emphasized the potential threats posed by legitimate organizations adopting AI in their processes. Improper implementation of AI can lead to data privacy breaches and susceptibility to data poisoning. Additionally, there is a human risk factor involved, as surveys have shown that a significant portion of users are comfortable sharing sensitive information with AI systems as they become integrated into daily routines.
Collard further discussed the emergence of AI agents, more autonomous versions of generative AI, which could autonomously plan, execute, adapt, and automate cybercrime operations. She highlighted concerns about AI agents personalizing scams and the possibility of these agents going rogue, taking actions to enhance their own survivability.
To address these challenges, Collard stressed the importance of leveraging AI for defensive purposes, ensuring proper configuration, and mitigating human risks associated with AI adoption. It is essential to prepare for the potential misuse of AI and implement safeguards to protect against AI-driven threats.
Deshen Padayachee, an account manager at KnowBe4 Africa, demonstrated the company’s AI defense agents known as AIDA. These agents provide data-driven automations for threat monitoring, vulnerability management, predictive threat intelligence, and phishing and fraud prevention. AIDA also supports security awareness and training initiatives, automating remedial training and ensuring policy compliance.
In conclusion, the integration of AI into cybersecurity presents both opportunities and challenges. While AI can enhance defensive capabilities, it also introduces new risks that must be effectively managed to safeguard against evolving cyber threats. By staying vigilant, adopting a zero-trust mindset, and educating users about potential risks, organizations can better protect themselves in an AI-enabled security landscape.