AI's real risk: Unauthorized access to sensitive data, not job losses

The root cause is often structural, stemming from companies rapidly connecting AI agents to their internal data for live workflows. Permissions that were once merely untidy for human users become dangerous when an AI agent can access everything simultaneously and provide summarized answers. This transforms the concept of shadow IT, as employees no longer need to manually copy sensitive files; an AI agent can retrieve and repackage information upon request. A single misconfiguration can lead to repeatable, large-scale leaks of confidential data such as salary bands, redundancy lists, or unannounced financial results.

Samantha Wessels, president of Box's EMEA business, stated that successful AI adoption hinges on building a foundation of trusted content and robust governance, rather than simply deploying more AI tools. She emphasized that companies achieving the best results are those that understand where their knowledge resides, who should access it, and what decisions AI agents are permitted to make. The future of enterprise AI, according to Wessels, lies in secure, portable context that can move across various AI tools.

The research indicates a near-unanimous agreement among leaders that strong permissions and access controls are essential for trustworthy enterprise AI, and that improved governance can accelerate innovation. The practical, albeit unglamorous, work of understanding what each AI agent can access, who sees its outputs, and where it operates is deemed more critical than model selection. Visibility is key, as organizations cannot govern what they cannot see, and current evidence suggests many lack this crucial oversight, particularly regulated firms for whom such exposures carry reportable consequences.