AI-Powered Social Engineering

For two decades, security-awareness training leaned on a comforting crutch: phishing emails were easy to spot because attackers wrote badly. Misspellings, stilted grammar, generic "Dear Customer" greetings — the tells were the defense. Generative AI has retired that crutch. An LLM produces fluent, idiomatic, context-aware text in any language, and it does so at scale, so the floor quality of every phishing campaign has risen to match what once required a skilled native-speaking operator. Worse, it personalizes: fed a target's public footprint, the model crafts a lure referencing real projects, real colleagues, and real timing. The generic blast has become the tailored spear, cheaply, by the million.

Then there is synthetic media. Deepfake voice and video collapse the last assumption holding up many fraud controls: that a familiar voice or face is proof of identity. Documented cases now include finance staff authorizing large transfers after a video call with what appeared to be their executives, and voice-cloned calls reproducing a specific person's speech from seconds of public audio. This is business email compromise evolved into business voice compromise — and it targets the same thing every social-engineering attack always has: human trust and the gaps in human process.

Notice what does not defend against this. Spotting bad grammar is useless against flawless text. Recognizing a trusted voice is useless against a clone. The defenses that survive are procedural and behavioral, and they survive precisely because they do not depend on perceiving the fake. Out-of-band verification: any consequential request — a payment, a credential, a change to banking details — gets confirmed through an independent, pre-established channel, defeating impersonation no matter how convincing it is on the original channel. Resistance to manufactured urgency: nearly every one of these attacks manufactures time pressure to short-circuit verification, so "this is urgent and confidential, do it now" should escalate suspicion, not compliance.

This is why AI has made awareness training more important, not less — but the curriculum has to change. The lesson is no longer "look closely at the email." It is "trust the process, not your perception, and verify anything that matters through a channel the attacker doesn't control."