Criminals are using LLMs to craft phishing, automate malware and clone voices. Defenders have new tools — but will they arrive fast enough?
Illustration by IMF Alpha editorial · Reviewed by Pedro Marini
The new front in cyber conflict
We crossed a line when artificial intelligence stopped being an academic novelty and became a routine drafting tool for criminals. Over the past year attackers have shifted from handcrafted scams to machine-generated campaigns that land faster, sound more convincing and scale with very little technical skill.
This is not theoretical. Security teams are seeing richer social-engineering lures, voice-cloned business-email compromise and code snippets produced by large language models that lower the barrier to cybercrime. Think of generative AI like a power saw — it speeds work whether you are building a house or breaking into one.
How this round is different
What’s interesting here is that these are not isolated improvements; they compound. Faster creation plus higher quality plus easy assembly changes the economics of attack.
Why defenders are not helpless — but it’s messy
There is reason for guarded optimism. Security vendors are adopting generative models for detection, triage and automated response. CrowdStrike and SentinelOne, for example, sell AI-driven endpoint detection that leans on behavioral signals instead of static signatures. Microsoft has pushed a security copilot aimed at SOC analysts to speed investigations.
That said, using AI for defense is not plug-and-play. Models hallucinate. They sometimes flag benign activity as malicious, or miss a clever evasion. The payoff is quicker investigations, yes — and a new pattern of false positives to manage. In practice, the story is messier than the marketing suggests.
Practical shifts companies must make
These are basic hygiene steps. They won’t stop everything, but they raise the cost for opportunistic attackers.
A few concrete examples
Small changes in tooling can shave weeks or months off an attacker’s development cycle. That matters.
Regulation, market response and the long view
Expect a two-track reaction. Regulators and enterprise security teams will push for tighter controls around model use with sensitive data. At the same time, criminal tooling will continue to commoditize — underground markets are already packaging AI-assisted social-engineering kits.
Historically, each step up in attacker tooling provoked an industry adjustment: spam led to filters, polymorphic viruses led to behavior-based detection. The difference today is tempo. AI accelerates both attack and defense adoption curves. Who moves faster — and who actually adopts basic hygiene — will largely determine the net effect.
My take as a technologist and watcher of markets
This is not only a cybersecurity problem; it’s a business-continuity and trust problem. Boards need to stop treating AI risk as purely technical. Insurers will reprice exposure, and vendors that can show credible, observable AI safety controls will be worth more. If you want a simple hedge, favor companies that pair model innovation with strict access governance and clear telemetry.
What to watch in the next 12 months
The ecosystem will adapt — it always does. But adaptation costs money, leadership and time. The same generative tools that help analysts write better reports also let criminals write more believable lies. The real question for CISOs is not whether to use AI, but how to use it without letting it redefine what reasonable security looks like.
As privacy rules bite and data costs spike, synthetic data startups and cloud giants are racing to replace real-world training sets. Investors should be selective.
From privacy gains to battery headaches, on‑device large language models force chipmakers, apps and regulators to rethink mobile AI—and investors to reassess winners.
As chip makers and developers push compact LLMs and NPUs into handsets, expect faster answers, tighter privacy—and a shakeup in how apps make money.