AI-Driven Threat Actors Are Reinventing Social Engineering Attacks and How Concentric Can Help

AI-Driven Threat Actors
January 18, 2025

|

Social engineering attacks are not new as cybercriminals often try to manipulate people into revealing sensitive information or granting unauthorized access. But with the rise of artificial intelligence (AI), these tactics have grown more sophisticated and dangerous. Today’s adversaries are using machine learning to personalize phishing emails, voice phishing (vishing) calls, and other exploits, making them harder to spot and more likely to succeed.

Traditionally, phishing emails followed a generic formula full of spelling errors, awkward grammar, and suspicious links. Now, threat actors employ AI-driven tools to sift through social media, public databases, and leaked credential dumps. They gather personal details about a target such as their workplace, job, hobbies, and even preferred communication style, to tailor emails that appear remarkably authentic. An email might look like it is from your boss, referencing specific internal projects or shared interests, prompting you to click on a malicious link or provide login credentials without a second thought. AI can also streamline attacks based on pretext to create a more convincing narrative that a relationship is already established and legitimate. It does this by generating a long email thread to make the scam seem more convincing. 

AI also is not just automating email attacks, it is elevating phone-based scams. Using advanced speech synthesis and deepfake audio, adversaries can replicate someone’s voice, often a high-level executive or trusted colleague, and call employees with requests that seem entirely legitimate. Victims may be asked to verify financial details or transfer funds. With these convincing deepfake calls, the old “check for strange grammar” rule simply does not apply, and the phone conversation feels real and urgent.

Machine learning capabilities also extend to social media manipulation with deepfakes and chatbots. In December 2024, there were numerous deepfakes shared across the surface web and the deep dark web. 

  • Users in the subreddit “r/Democrat” shared AI-generated images and videos of U.S. President-Elect Donald Trump and Elon Musk. They depict Trump and Musk in various degrading scenarios. 
  • Additionally, multiple X accounts have shared fake videos likely created by the Russian disinformation network “Storm-1679” (also known as “Matryoshka”). The videos used AI-generated voice-overs to impersonate researchers and scholars calling for an end to sanctions against Russia or maligning Ukrainian immigrants. 
  • Threat actors also deploy bots that can engage with employees or the public in real time, studying interaction patterns to make their language and tone more persuasive. In some cases, these bots will impersonate customer support agents or other trusted figures, tricking users into surrendering personal data.

How Concentric Can Help

With growing AI-powered attacks quickly adapting to detection methods, organizations need to move beyond traditional security tools. Email filters based solely on keywords or suspicious domains may be easily bypassed by AI-driven phishing attempts. Traditional filtering found in email providers to block spam and phishing emails are being circumvented also using AI and bad actors are using AI to get around safeguards which ensures a successful email delivery. Likewise, basic employee training must be consistently updated to cover tactics like deepfake voice calls or realistic chatbots. Concentric recommends the following steps to protect employees and data in this environment:

  • Educating employees on the latest attack methods, including deepfake calls and AI-created impersonations; 
  • Regular simulations can help staff detect subtle red flags; 
  • Having a clear plan to contain and investigate breaches quickly can also help to limit any damage AI-driven attacks might inflict on personnel or corporations.

Our intelligence and cyber teams are here to help if you would like to learn more about AI-powered attacks and how to best prevent and stop them from impacting you, your family, and your business. 

Share this post:

Facebook
Twitter
LinkedIn