Confluent cofounder warns of AI's 'dark side' amid safety fears

Kreps highlighted that many beneficial AI applications have a corresponding 'dark version.' For instance, an AI system that excels at coding could also be used for sophisticated hacking, and AI capable of designing new drugs could potentially be used to develop undetectable poisons. He emphasized that these concerns are not a sign of neuroticism or pessimism but rather common sense.

His remarks come amid a series of unnerving headlines related to AI safety. Recently, Jacob Coxon, an Anthropic researcher, resigned, warning that AI labs are 'gambling with our lives.' Similarly, Geoffrey Hinton, a prominent AI researcher, suggested that there is a non-negligible chance AI could lead to human extinction within a decade.

These warnings have faced skepticism from some prominent figures in the AI field, including Elon Musk, who has described the concerns as a potential 'setup' or 'psy op.' However, Kreps maintains that mocking those who express worry about AI risks is unhelpful, especially without offering substantive solutions for mitigating these problems. He expressed confidence in the industry's ability to develop effective guardrails, but cautioned that this has not yet been fully achieved.