Anthropic co-founder Chris Olah spoke at the Vatican, emphasizing the need for external oversight of AI development. He highlighted the conflict between profit motives and ethical AI, arguing that institutions like the Catholic Church, scholars, and governments must supervise the industry to maintain moral focus.
Chris Olah, a co-founder of the AI company Anthropic and an AI safety researcher, spoke at the Vatican alongside Pope Leo XIV. Olah, who leads interpretability research at Anthropic, acknowledged the unusual nature of his presence at an event focused on the dangers of AI. He stated that AI companies must ensure they are 'doing the right thing' despite profit incentives and external pressures. Olah argued that outside critics, including the Catholic Church, scholars, and governments, must supervise the AI industry to keep its moral obligations in focus, countering the idea that only computer scientists should handle AI matters. Olah previously worked at Google Brain from 2015 to 2018 and led OpenAI's interpretability team from 2018 to 2020. He left OpenAI in 2020 due to AI safety concerns to co-found Anthropic. Anthropic was recently valued at $965 billion and has confidentially filed for an IPO. Olah's net worth is estimated at nearly $8 billion. His views contrast with techno-optimists like Marc Andreessen but align with Anthropic's mission of prioritizing AI safety. Pope Leo XIV's encyclical, 'Magnifica Humanitas,' also calls for a measured approach to AI development.
The call for external AI oversight from a prominent AI company co-founder at the Vatican underscores growing concerns about the ethical development and governance of artificial intelligence, potentially influencing regulatory approaches and public perception.