The past two years felt like a sprint — models that lived in papers and demos are now running on developers' machines. That was obvious. What’s slipping under the radar is a quieter, more consequential shift: the same open-source LLMs that make innovation cheap and local are being repurposed by attackers to write phishing, automate reconnaissance, and even sketch evasive malware.
Why this matters now
Open models are trivial to run on a laptop, inexpensive to fine-tune, and awkward to control once they’re public. Put those three together and you get a multiplier for attackers. Where a capable operator used to craft a single bespoke phishing campaign, scripts can now spit out thousands of tailored messages, brainstorm plausible domain squats, or tweak payloads to dodge signature-based antivirus. It is not hypothetical. Think back to the mid-2000s: Metasploit and exploit kits turned cybercrime from boutique work into a scalable service. Open LLMs feel like the next toolbox — only faster, because a natural-language prompt now produces human-sounding lures, and code-synthesis features collapse the time needed to prototype malicious code.
How attacks are shifting
- Social engineering at scale. AI drafts highly contextual spear-phishing that folds in job title, local events, even corporate tone. It reads like an inside email.
- Better evasion. Models suggest polymorphic tweaks, packing strategies, and obfuscation patterns that can slip past heuristic scanners.
- Automated reconnaissance. Chat-style agents scrape and synthesize OSINT to map targets and point to obvious intrusion vectors.
- Lowered bar to entry. Novice actors can field convincing campaigns without deep technical chops.
Why defenders and investors should care
In the near term this pushes demand away from signature-only controls and toward behavior- and intent-based detection. Expect endpoint detection, extended detection and response (XDR), and cloud workload protections to get more attention. Identity becomes a pressure point too — phishing-resistant MFA and faster patching matter more than ever.
For investors, that means watching vendors that actually have telemetry and response breadth rather than vendors selling one-off point products. Pure-play security firms focused on behavioral analytics should stay interesting, and cloud providers that can bake AI-aware protections into their platforms will have an edge. Don’t forget the compute suppliers; GPUs and chipsets still matter because they power model training and inference.
A bit of nuance
Open models cut both ways. Security teams are already using the same tools to triage alerts, draft signatures, and model attacker playbooks. There’s a moral and policy puzzle here: restricting access may slow some attackers but it slows defenders and researchers too. Standards, export controls, and provenance requirements will matter, but market forces are shaping short-term responses more quickly than regulation will.
Practical steps for CISOs
- Treat AI-enabled social engineering as a top threat vector; change phishing simulations so they mimic these higher-quality lures.
- Invest in behavioral detection and richer telemetry, not just more signature rules.
- Harden identity: require phishing-resistant MFA and tighten remote access.
- Push vendors for model governance and attestations when AI is in the supply chain.
What to keep an eye on
- Regulatory guidance around model provenance and export controls.
- Product moves from XDR and SIEM vendors that promise AI-native detection.
- Shifts in security budgets toward detection and identity — you’ll see that in vendor earnings and capex.
My read is we’re entering another cyclical arms race. Attackers will weaponize convenience; defenders will respond with telemetry-rich, AI-aware protections. For investors, the task is to separate firms that genuinely own telemetry and response from those selling clever-sounding one-offs. For security leaders, the practical play is to double down on identity, telemetry, and rapid detection. The same tools that democratized innovation have also democratized threatcraft. Meeting that means better signals and smarter response, not just louder alarms.