Microsoft Threat Intelligence recently intercepted a sophisticated phishing campaign that appears to have leveraged AI-generated code to avoid detection. The attack targeted organizations in the United States and attempted to hide its malicious payload inside an SVG file disguised as a PDF.

On August 18, attackers used a compromised small business email account to distribute phishing messages. The emails were self-addressed, with actual targets hidden in the Bcc field, and mimicked file-sharing notifications. The attached file, named “23mb – PDF- 6 pages.svg,” contained embedded scripts that redirected recipients to a fake CAPTCHA page, likely leading to a fraudulent login form.

AI-Powered Obfuscation Techniques

The SVG file was notable for its unusual obfuscation style. Instead of traditional encryption, the attackers disguised their payload using business-related terminology. Invisible elements simulated a business dashboard, while terms such as “revenue,” “operations,” and “risk” were encoded as hidden attributes. Embedded JavaScript then converted these terms into malicious instructions for redirecting browsers and tracking sessions.

Microsoft’s Security Copilot analyzed the code and concluded it was likely generated by a large language model, citing telltale signs including:

• Overly descriptive function names with random suffixes
• Modular, over-engineered code blocks
• Verbose and generic comments in formal business language
• Formulaic obfuscation techniques
• Unusual CDATA and XML usage

“This type of code is unlikely to be written manually,” Microsoft noted, highlighting the sophistication and verbosity introduced by AI.

How Microsoft Stopped the Attack

Despite the advanced obfuscation, Microsoft Defender for Office 365 detected and blocked the campaign by analyzing patterns in infrastructure, delivery, and message context. Key indicators included:

• Self-addressed emails with hidden BCC recipients
• Uncommon SVG file format disguised as a PDF
• Redirected domain already linked to phishing
• Suspicious network activity, such as session tracking and browser fingerprinting

These combined signals allowed Microsoft to halt the campaign before any damage occurred.

Key Takeaways for Organizations

AI-generated phishing attacks may produce more sophisticated and polished code, but they also leave detectable artifacts. Microsoft recommends organizations adopt the following measures to mitigate similar threats:

• Enable Safe Links in Microsoft Defender for Office 365
• Activate Zero-hour Auto Purge (ZAP)
• Use phishing-resistant authentication methods
• Turn on cloud-delivered protection in antivirus solutions

This incident underscores a growing trend: both attackers and defenders are increasingly leveraging AI. While cybercriminals use AI to enhance their attacks, advanced security systems remain capable of identifying and neutralizing AI-aided threats before they can cause harm.

Source: https://www.infosecurity-magazine.com/news/ai-generated-code-phishing