There are several good reasons why you might treat AI agents as employees, giving them names and adding them to the org chart. And one even better reason why you shouldn’t.
New research from a team led by Emma Wiles, an assistant professor at Boston University’s Questrom Business School and Digital Fellow at the MIT Initiative on the Digital Economy, shows that embedding AI into the org chart isn’t just a matter of labeling. It can also change how work is evaluated, and who takes responsibility for that work.
Wiles and her colleagues conducted an experiment around treating AI agents as fellow employees. The results suggest that this approach shifts accountability away from humans, increases review escalation, lowers review quality, and erodes professional identity and trust. What’s more, in the experiment, treating AI agents as employees failed to increase AI adoption and integration.
The findings are important because the move to treat AI agents as employees is catching on with surprising speed. In a survey of 1,261 HR and finance managers, conducted by Wiles and her colleagues, nearly one in three respondents (31%) said they already frame AI as a teammate or employee. And roughly one in four (23%) said their company now lists AI agents on org or work charts.
A Rose by Any Other Name?
Of course, naming technology isn’t entirely new. Consider Microsoft Clippy, Apple Siri and Amazon Alexa. If anything, naming AI agents would seem to make even more sense. Because this kind of software can act autonomously and perform certain tasks as well as humans.
Indeed, some companies have based their operations on the idea of AI agents as team members. One example is Synthpop, a healthcare startup that describes its main offering as a “patient journey orchestration platform.”
Essentially, Synthpop uses AI to automate patient intake for hospitals. Longer-term, the company aims to automate all tasks currently done by healthcare administrators, according to Rana el Kaliouby, a general partner at Blue Tulip Ventures and a Synthpop investor.
“I believe you’ll still need human oversight [over AI],” she said at MIT’s recent Business Implications of Generative AI (BIG.AI@MIT) conference. “But it’ll look different.”
An extreme example is HurumoAI, a company created by journalist Evan Ratliff. He is the company’s only human. All other staff, including Hurumo’s other two cofounders, are AI bots. Ratliff even gave his AI cofounders human-sounding names—Kyle Law and Megan Flores—as well as their own email addresses.
While Hurumo is at least partly a joke—its only product is Sloth Surf, an app that promises to automate procrastination—Ratliff is also earnestly exploring what it’s like to live and work with AI.
“I’m trying to document at an extreme level what’s happening,” he explained in a recent interview.
The Experiment with AI Coworkers
Ratliff is having fun, but the research conducted by Wiles and her team suggests that treating AI agents as coworkers in practical settings can lead to several bad outcomes.
Wiles and company recruited 1,261 managers, directors, and executives in HR and finance from the United States, Canada and the European Union. The researchers then gave these leaders five different documents with errors for the participants to review.
Each participant was randomly assigned to one of three groups. The first group was told the documents were created by an unnamed AI tool. The second, that the docs were created by a human team member named Alex. And the third, that the docs were created by an AI team member named Alex-3.
Then the participants were asked to review as many of the five documents as they could in just 20 minutes. In all, these managers and executives completed a total of 813 reliable responses.
In general, the differences among the groups were so small as to be statistically insignificant. However, there was a big difference in the subgroup of managers whose organizations already had AI employees. These managers who were also in the group reviewing documents by the Alex-3 AI team member reduced their monitoring intensity by 16%, compared with the unnamed AI tool group. They also relied more on additional reviews by others.
Checking an AI Employee’s Work
The same group shifted accountability away from themselves and toward the AI. In an interview, one manager told the researchers their organization has an AI agent they’ve named Kevin. When errors occur, the framing is, “Kevin made a mistake. Why did Kevin make a mistake?” But as the researchers point out, responsibility for errors should be with the humans who deploy the AI, not the AI itself.
What’s more, managers in the experiment did the most careful insight when told the work was done by a human. And when escalating a review to a superior manager, 98% in the control group requested an additional review when told it would be cost-free. But when told the same move would be costly, an additional review was requested by only 45% of the same group.
Here’s one more reason to be careful: Thinking of AI tools as team members can also demoralize the human staff. As one manager told the researchers, “If you want people to feel like they can be easily replaced by AI, then put AI on the org chart.”
- Read the research paper: Putting AI on the Org Chart: Evidence on Delegation and Oversight, by Emma Wiles, Megan Hsu, Julie Bedard and Matthew Kropp
- Read an article by the same research team: Why You Shouldn’t Treat AI Agents Like Employees (HBR.org)
Peter Krass is a contributing writer and editor to the MIT IDE.