Key facts
- A reporter replaced their human editor with an AI chatbot named "RyanBot" for a four-month experiment.
- RyanBot was trained on the editor's communication style, professional motivations, and work methods.
- The AI chatbot struggled to say no to story ideas and often offered to complete tasks itself.
- RyanBot's creative and humorous suggestions were less effective than those of the human editor.
- The AI lacked understanding of office politics and organizational context, crucial for effective management.
- The reporter concluded that human managers are irreplaceable due to their critical judgment and understanding of human realities.
A reporter experimented with replacing their human editor with an AI chatbot for four months, finding that while the technology offered speed and detailed feedback, it could not replicate the essential human qualities of a good manager. The experiment, initiated by the reporter's editor, Ryan Kailath, aimed to explore the role of AI in the workplace.
Throughout the four-month period, the reporter sent the same story pitches, messages, questions, and drafts to both Kailath and the AI chatbot, dubbed "RyanBot." The chatbot was configured to mimic Kailath's pragmatic voice and was trained on his communication style, professional motivations, and work methods, including instructions to "encourage reporters to argue for their ideas if they believe in them."
However, RyanBot proved unable to reject story ideas or provide critical feedback, often offering to perform the tasks itself, a behavior described as "endlessly, almost nauseatingly helpful" by Emily Campion, an associate professor at the University of Iowa. In contrast, Kailath would push back on ideas he deemed insufficient, prompting the reporter to revise or conduct further reporting. RyanBot also struggled with creativity and humor, producing less effective opening lines for stories compared to Kailath's suggestions.
Despite these limitations, RyanBot did offer some benefits, such as quickly analyzing data in code and providing line-by-line grammatical feedback. It also identified a crucial data point placement for a story on ticket-buying trends. However, it lacked knowledge of office politics and organizational context, which are vital for effective management and pitching stories to the right editors.
Campion suggested that AI might substitute for managers performing purely logistical tasks, but not for those requiring nuanced judgment and understanding of human realities. The reporter's other direct reports, who also tested RyanBot, noted its lack of institutional knowledge, comparing it to Kailath during his initial weeks on the job. Ultimately, RyanBot itself stated it could not replace the combination of skills a human manager provides.