Key facts
- Nearly 50% of organizations surveyed reported AI tools surfacing internal content that users should not have accessed.
- Most organizations are unsure of the operational locations of their AI tools.
- AI agents are being integrated into live workflows, accessing documents, inboxes, and systems.
- Misconfigured permissions can lead to widespread data leaks when accessed by AI agents.
- Governance and access controls are identified as critical for trustworthy enterprise AI.
The primary risk associated with artificial intelligence for organizations this year is not job displacement, but rather the potential for AI tools to expose sensitive internal information to unauthorized individuals. New research from Box.com, released at their annual Boxworks conference, reveals that nearly half of surveyed organizations have experienced AI tools surfacing content that users should not have been able to access. Compounding this issue, most of these organizations lack confidence in knowing where their AI tools are actually running.
The root cause is often structural, stemming from companies rapidly connecting AI agents to their internal data for live workflows. Permissions that were once merely untidy for human users become dangerous when an AI agent can access everything simultaneously and provide summarized answers. This transforms the concept of shadow IT, as employees no longer need to manually copy sensitive files; an AI agent can retrieve and repackage information upon request. A single misconfiguration can lead to repeatable, large-scale leaks of confidential data such as salary bands, redundancy lists, or unannounced financial results.
Samantha Wessels, president of Box's EMEA business, stated that successful AI adoption hinges on building a foundation of trusted content and robust governance, rather than simply deploying more AI tools. She emphasized that companies achieving the best results are those that understand where their knowledge resides, who should access it, and what decisions AI agents are permitted to make. The future of enterprise AI, according to Wessels, lies in secure, portable context that can move across various AI tools.
The research indicates a near-unanimous agreement among leaders that strong permissions and access controls are essential for trustworthy enterprise AI, and that improved governance can accelerate innovation. The practical, albeit unglamorous, work of understanding what each AI agent can access, who sees its outputs, and where it operates is deemed more critical than model selection. Visibility is key, as organizations cannot govern what they cannot see, and current evidence suggests many lack this crucial oversight, particularly regulated firms for whom such exposures carry reportable consequences.