Source link : https://tech365.info/your-builders-are-already-operating-ai-regionally-why-on-device-inference-is-the-cisos-new-blind-spot/
For the final 18 months, the CISO playbook for generative AI has been comparatively easy: Management the browser.
Safety groups tightened cloud entry safety dealer (CASB) insurance policies, blocked or monitored visitors to well-known AI endpoints, and routed utilization by sanctioned gateways. The working mannequin was clear: If delicate information leaves the community for an exterior API name, we are able to observe it, log it, and cease it. However that mannequin is beginning to break.
A quiet {hardware} shift is pushing giant language mannequin (LLM) utilization off the community and onto the endpoint. Name it Shadow AI 2.0, or the “bring your own model” (BYOM) period: Workers operating succesful fashions regionally on laptops, offline, with no API calls and no apparent community signature. The governance dialog remains to be framed as “data exfiltration to the cloud,” however the extra speedy enterprise threat is more and more “unvetted inference contained in the system.”
When inference happens locally, traditional data loss prevention (DLP) doesn’t see the interaction. And when security can’t see it, it can’t manage it.
Why local inference is suddenly practical
Two years ago, running a useful LLM on a work laptop was a niche stunt. Today, it’s routine for technical teams.
Three things converged:
Consumer-grade accelerators got serious: A MacBook Pro with 64GB unified memory can often run quantized 70B-class models at usable speeds (with practical…
—-
Author : tech365
Publish date : 2026-04-12 16:17:00
Copyright for syndicated content belongs to the linked Source.
—-
1 – 2 – 3 – 4 – 5 – 6 – 7 – 8