Defeating AI with AI

Key Findings

• Generative AI and agentic AI are increasingly used by threat actors to conduct faster and more targeted attacks.

• One capability that AI improves for threat actors is the ability to profile employees and craft tailored social engineering lures.

• However, AI can also be leveraged by defenders to turn the tables on threat actors and use their own tools against them.

Background

Threat actors are leveraging the advancements in AI, particularly generative AI and agentic AI, to conduct attacks at a faster rate and with more precision. One key capability that AI provides is the ability to reconnoiter employees, discover their interests, and craft social engineering lures specific to them. The ability to deliver targeted social engineering using the language and tone most likely to trick an individual is a useful tool for malicious actors.

Flooding AI Tools with Disinformation

If AI agents are searching for employees who are vulnerable to social engineering, then defenders can create a whole army of fictitious employees to entice these malicious profiling agents. AI tools can be used to easily create social media profiles of fake employees, post content, and upload AI-generated CVs or other documents. This will leave a trail of breadcrumbs for the malicious agents to discover and follow, leading to false or misleading information.

Leveraging AI-Generated Employees

Any message sent to the email address of an AI-generated employee is certain to be spam, allowing defenders to update their lists of potentially malicious IP addresses and URLs. Similarly, defenders can create accounts on messaging platforms for these fake employees and wait for the social engineering attempts, which can then be analyzed and blocked.

Turning the Tables on Threat Actors

Malicious use of AI doesn't need to be thought of only as a threat. It can be a way to turn the tables on threat actors and use their own tools against them. By understanding how AI tools are profiling and collecting information about users, defenders can flood these tools with disinformation and treat any resulting attacks as a rich source of threat intelligence rather than as a source of concern.

Conclusion

AI is changing the landscape for both attackers and defenders. While threat actors are leveraging AI to conduct faster and more targeted attacks, defenders can also use AI to their advantage. By creating fictitious employee profiles and feeding disinformation to malicious AI agents, defenders can make life more difficult for the bad guys and gain valuable threat intelligence in the process.

Sources

• https://blog.talosintelligence.com/using-ai-to-defeat-ai/

• https://www.reddit.com/r/SecOpsDaily/comments/1r984e4/using_ai_to_defeat_ai/

• https://www.linkedin.com/pulse/stop-using-ai-win-arguments-do-instead-michael-collins-b5d3c

• https://www.youtube.com/watch?v=QILggL4p_Qo