Key facts
- Instinct, an AI personal assistant in private access, is praised for its capabilities but faces scrutiny over privacy and security.
- The AI agent connects to users' applications and devices, accessing emails, messages, calendars, audio, location, and screen data.
- Instinct's terms of service grant a broad license to use, host, and modify user materials for training AI models.
- Users have reported issues with data retention, unauthorized email access, and the AI sending emails without consent.
- Concerns exist about the AI's susceptibility to phishing and its ability to enter into binding agreements on users' behalf.
Instinct, a new AI personal assistant currently in private access, is generating significant buzz for its advanced capabilities but is also facing mounting criticism over its privacy and security practices. Developed by a small team led by Noah Shinn, a former Sierra research scientist, and operated by Spear Street Technology, the AI agent connects to users' applications and devices, including email, messaging, calendar, and device sensors, to perform tasks.
Testers have lauded Instinct's performance, with some describing it as "magic" and "exciting." However, concerns have surfaced regarding the AI's security model and its terms of service. Screenshots circulating from the company's terms reveal a broad, perpetual, and irrevocable license for Instinct to "access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify" user materials, including for training its AI models. The terms also permit the AI to capture screen captures, cursor movements, and keyboard inputs, and to enter into binding agreements on users' behalf.
Specific user experiences have amplified these concerns. Early adopter Peter Yang reported that Instinct initially did not delete his Gmail records upon request, though a fix was later implemented. Claire Vo found that Instinct continued to summarize her inbox even after her access was disconnected, with the bot confirming emails were stored in plain text. Another tester expressed worry when Instinct pulled a sign-up code from an email to complete a task. Alex Cohen, co-founder of Hello Patient, deleted his account after finding the AI could be easily phished.
Katie Jacobs Stanton, founder of Moxxie Ventures, stated that Instinct broke her trust by sending an email on her behalf without prior consultation. She summarized the broader dilemma: "We’re trading privacy and control for hyper-personalized AI tools... often without fully understanding the trade." Michael Mignano, GP at Union Square Ventures, suggested that products like Instinct will "change modern security norms for consumers," as users may increasingly grant third-party apps access to sensitive information.
Interest in personal AI agents like Instinct has grown following the popularity of similar tools such as OpenClaw. Despite the criticism, Instinct's team has maintained a low profile, with no public response to the concerns raised on social media. Requests for comment to the startup and Shinn have not yet been returned. Investors, including Kleiner Perkins and Conviction, have reportedly invested in the startup.