Key facts
- OpenAI released its new Astra AI model, described as its most powerful and capable to date.
- Astra employs a reasoning technique known as "opaque recurrence" or "recurrent depth."
- This technique may obscure the AI's "chain of thought," making its decision-making process harder to monitor.
- AI safety experts are concerned that reduced monitorability could lead to misalignment and a "race to the bottom."
- OpenAI claims Astra has enhanced cybersecurity capabilities and is the "best model for software engineering to date."
- The model is being made available to OpenAI customers and through paid plans and its API.
OpenAI has launched Astra, its latest and reportedly most powerful AI model, which incorporates a novel reasoning technique called "opaque recurrence" or "recurrent depth." This method has raised significant concerns among AI safety experts due to its potential to obscure the model's "chain of thought," a critical process for auditing AI decision-making.
AI safety advocates worry that this reduced monitorability could undermine efforts to ensure AI alignment and potentially lead to a "race to the bottom" among AI labs. Buck Shlegeris, CEO of Redwood, voiced extreme concern, suggesting that further development could destroy chain-of-thought monitorability. Zvi Mowshowitz, an AI safety advocate, proposed that legal intervention might be necessary to prevent such a scenario.
OpenAI, however, has downplayed these concerns, stating that Astra's use of opaque recurrence will be limited and that its chain of thought will remain legible. Chief scientist Jakub Pachocki acknowledged that increasing model capabilities make monitorability more challenging, especially when models use fewer language tokens. OpenAI president Greg Brockman suggested that the company may have reached artificial general intelligence (AGI), describing it as a "mission concept."
Astra is being made available to OpenAI customers, including those using its cybersecurity program Daybreak, and will roll out to paid subscribers and via API. OpenAI claims Astra possesses advanced cybersecurity capabilities, including identifying zero-day exploits, and is the best model to date for software engineering.