- MIT research into AI productivity gains paint a more nuanced picture on how workers are spending their time when using AI tools.
- Companies seeking to replace workers with AI may find it’s not that simple. Automating tasks is already happening, but widely replacing workers with AI is likely cost prohibitive.
- To achieve the best results, agentic AI design should focus on collaboration, where human workers are trained to use AI as an assistant, rather than all-knowing.
Are we using AI the right way? Are we getting the most out of our AI tools? How can we get workers on board with using AI if they’re afraid it will replace them?
As companies look to expand the use of AI tools and optimize work, one of the biggest challenges is parsing what’s feasible with AI, and what’s just hype.
In a recent MIT CSAIL Alliances podcast, Neil Thompson and David Autor — both MIT researchers who study AI’s impact on work and productivity — took an evidence-based look at what’s actually happening with AI in workplaces today. Thompson, director of MIT FutureTech and research lead at MIT Initiative on the Digital Economy, and Autor, an economist, co-author of The Work of the Future, and IDE research team member, draw on a growing body of research to examine whether AI is meeting expectations, where implementation is falling short, and what the data suggest about getting better results.
Get MIT IDE insights in your inbox. Subscribe to our newsletter →
AI productivity gains are smaller than you think
“A number of people have done work where they’ve tried to estimate how much more productive [AI] is making people, and the answers are pretty small,” said Thompson.
The reason comes down to a problem most organizations aren’t measuring: overhead. Thompson cited research by economist Anders Humlum that found roughly half of the time people spend using large language models goes toward managing the AI itself, such as quality control, bias checks, prompt refinement. While productivity increases with AI tools in one sense, the efficiency gains get eaten up by new managerial tasks.
Thompson pointed to a detailed study by that Metr showed how this plays out in software development, a field frequently cited as proof of AI’s productivity potential. The study tracked programmers through the full process of updating a software library, including not only coding but all the steps involved.
“Even though the people themselves were like, ‘I’m 20% more productive,’ they were actually 20% less productive overall if you actually measured it carefully,” said Thompson.
“This is not that I think that everywhere is going to be made less productive by AI,” he added. “but I do think what it is telling us is there are going to be a bunch of frictions that are going to come in as we do it.”
Companies that have succeeded, said Thompson, have taken the time to look at their processes, experiment with prompts, and build in quality checks to make it work for them.
Will AI replace your job?
The dominant fear around AI is job replacement, but completely replacing workers using AI is more challenging than people may think, says Thompson, especially if the work has to be accurate.
“I want to emphasize how important this is about how we expect AI to play out,” said Thompson. “One of the things about these AI systems is getting the last little bit of performance is incredibly expensive…if you need a system to be almost perfect, it’s very, very costly.”
Rather than eliminating roles or departments wholesale, AI works best when it automates supporting tasks within a job, making the remaining human work more consequential and requiring higher levels of judgment. Work may get done faster, but it will still require human oversight.
Autor noted that at Meta, a similar pattern has emerged in software engineering. Despite widespread concern about AI eliminating programming jobs, the company is still hiring programmers, although in more senior positions that require advanced expertise.
The problem of the “missing middle” and career development in the era of AI
The demand for advanced expertise raises concerns about training up entry-level and lower-skilled workers. If companies are only hiring for senior roles, where do early career workers go and how do they get the skills needed to advance?
It’s also an organizational challenge for companies. If AI increases the need for expertise, how do you develop the next generation of workers who haven’t had the chance to gain that knowledge?
“We have lots of programs to get people skills before they enter the labor force: residencies, apprenticeships, being a junior lawyer. I think this is a solvable problem. I do not think we are in a world where the ladder is being pulled up and no one can reach the first rung anymore,” Autor said.
However, ensuring proper training and career advancement requires intentional investment.
AI agent design must be collaborative
So AI isn’t going to replace your whole workforce, but it can lighten the load by automating workflows, right? The advancement of AI agent technology has exciting potential for streamlining processes, however creating a system that lets you hand a task off to a machine and walk away can be challenging and costly.
“Most people think it’s an automation tool,” says Autor. “We’ll take what we’re doing and then the machine will do it for us better. Done. But actually, automation is extremely hard.”
True automation requiring zero human intervention is rare and expensive. Most useful tools don’t do the work for you, but help you do it better. For example, a stethoscope or a chainsaw, when placed in the right hands, amplify a worker’s expertise. AI, the research suggests, works the same way.
When it’s treated otherwise, the results can be worse than doing nothing at all.
Autor used CheXpert, a Stanford AI system designed to analyze chest X-rays, as an example of this. On its own, it performed as well as two-thirds of radiologists at reading scans but when radiologists used it as a decision-support tool, their performance actually declined.
The system wasn’t designed for collaboration, said Autor. When a doctor and the AI disagreed, the doctor tended to defer to the machine, even in cases where the doctor’s instincts were right. There was no mechanism for the two to reason through disagreement together.
“It’s not a limitation of AI,” Autor notes. “It’s a challenge of designing it in a way that collaborates effectively with human capacities.”
How to build a better AI productivity framework
According to Autor and Thompson, the companies getting the most from AI are the ones asking better questions before they deploy.
Account for the full cost
“Many firms are eager to get into this space and say we want to invest in AI, and then they find it’s actually a lot more work. It’s making people just a little more productive,” he said. Before declaring a win, he argues, you have to measure the whole workflow. Time spent on overhead and managing AI tools must be accounted for as well.
Invest in human expertise alongside AI
If AI raises the bar on what’s expected of your people, developing that expertise isn’t a soft priority, it’s a core business strategy. The organizations that neglect their workforce pipeline will find themselves with powerful tools, yet a lack of skilled workers to use them well.
Design for collaboration, not automation
The instinct to hand tasks off entirely to a machine and walk away is where most implementations run into trouble. The question isn’t just what AI can automate, but what it should — and whether the conditions exist to do it well.
The data on AI in the workplace is still emerging, but the early findings are consistent enough to offer a clear signal: the gap between AI’s promise and its results isn’t closing on its own. It closes when organizations are willing to let the evidence challenge their assumptions — and build from there.