- A new whitepaper argues that every task you delegate to an AI agent shapes who you become, not just what gets done.
- AI agent alignment is the wrong goal, author Michael Schrage argues. Instead he proposed a framework to keep human judgment in the loop.
- By adopting two new tools—an Agent Charter and Reflective Escrow—organizations can retain more agency and decide when AI should act, ask, or step back.
Effectively integrating AI agents into everyday workflows has become an enterprise priority. From coding to customer service, intelligent agents promise faster, more efficient and frictionless processes. But as organizations expand these capabilities, they’re also increase how much decision-making—and human agency—they effectively delegate to machines.
Michael Schrage, a lecturer at MIT Sloan and Digital Fellow at the MIT Initiative on the Digital Economy, argues in a new whitepaper, Whose Ally is ‘Your’ Agent, that the real question isn’t how well an agent aligns with what you want today—it’s whether it helps you become a person who can choose a better tomorrow. Every decision delegated transfers a small amount of human agency—not just work—to a machine. Does that make human capital more valuable or less?
“Every agentic interaction strengthens some human capabilities while allowing others to atrophy,” said Schrage. “Every delegation is a choice about which forms of judgment get practiced, which get outsourced, and ultimately who people—and organizations—become.”
The emerging strategic and operational challenge is no longer simply what agents can do, but what human capabilities they cultivate—and which they replace. Handing ever-greater agency to ever-smarter AI agents may improve efficiency while quietly weakening the judgment people need to exercise themselves. The question is not just whether AI makes work smarter. It is whether working with AI makes humans smarter.
AI Agents and Our “Selves”
Instead of treating agents as default decision-makers, Schrage asks whether they should become explicit allies to different forms of human judgment—strategic, disciplined, exploratory or impulsive—rather than tacitly serving whichever preference happens to issue the prompt.
Drawing inspiration from Nobel laureate Thomas Schelling’s classic 1978 paper, “Egonomics, or the Art of Self-Management,” Schrage argues that people contain competing interests. A CFO preparing for an earnings call may have a strategic self that wants to challenge assumptions and preserve optionality, but also an exhausted midnight self that simply wants seventeen open decisions resolved before going home. An AI agent told to “clear the queue” may efficiently serve the second self at the expense of the first. Most AI systems currently ally with whoever is typing the prompt at that moment. Schrage argues that this default deserves explicit governance rather than implicit acceptance.
The deeper point goes beyond a rational long-term self that’s battling an irrational short-term one. Different selves can represent legitimately competing objectives: speed versus deliberation, exploration versus commitment, generosity versus discipline, present performance versus future capability. Agentic AI does not eliminate these conflicts. It acquires power within them.
“Whose ally is the agent, and on what authority?” Schrage asks. “Which self runs the show? How do we know?”
How to Build an AI Agent for Better Judgment
Consequently, the most challenging frontier for agentic AI isn’t greater autonomy, it’s better discernment—knowing when an agent should act, when it should ask, when it should defer, and when it should deliberately slow the human down.
Relying on data input creates a challenge Schrage likens to the Quantified Self movement of the 2010s. During that era, people tracked their outputs—step counts and sleep data—turning mundane tasks and exercise into a game. The measurements were precise, but they largely ignored the intentions, trade-offs, and motivations that gave those numbers meaning.
“Ten thousand steps taken by an exhausted, avoidant self are hardly equivalent to ten thousand taken by a disciplined, strategic self,” Schrage writes. “The movement had no theory of which self was being quantified.”
To overcome this, Schrage proposes an alternative paradigm: a Quantified Selves architecture built on two connected instruments.
The first instrument is an Agent Charter that specifies which human interests the agent is authorized to represent, what decisions it may make independently, and what it must escalate for human review. The second is Reflective Escrow: Like a financial escrow, it temporarily holds both a proposed decision and the reasoning behind it before action is taken. The goal is clearly making visible which “self” the agent represented and why. Together, the charter defines authority while Reflective Escrow creates accountability. Human review becomes both governance and training data for improving future human-agent collaboration.
Over the next decade, Schrage predicts a widening divide. One group will increasingly use AI to minimize or even eliminate deliberation. The other will intentionally use AI to further cultivate and improve it. Competitive advantage may come less from having better agents than from developing better humans working with them. That is the bet, says Schrage.
He noted companies are already exploring this idea, comparing Reflective Escrow to Anthropic’s new reflection feature, released for Claude users in July. The feature helps people see what they used AI for and how it shaped their real-world actions. Adding Reflective Escrow to workflows, Schrage said, is a way for companies to find and build the right checkpoints for human judgment.
Where Is Agentic AI Heading?
Beyond Schrage’s work, how to optimize human collaboration with AI is a topic a number of researchers at the IDE are exploring.
- John Horton and Peyman Shahidi have studied when to add human intervention to agentic tasks, finding that grouping tasks together—with human input at the start and end—maximizes efficiency.
- At the IDE Annual Conference, Renee Gosline shared research that found that well-placed friction improves outcomes at little cost to efficiency.
- Sinan Aral’s Applied AI research group has found that matching people and AI by personality improves outcomes.
Where these researchers are analyzing AI use in practice, Schrage’s approach is theoretical. But the challenge of getting agents to exercise the right judgment is not. Alignment is the wrong goal, he argues. An agent perfectly aligned to what you wanted yesterday can quietly lock you into being that person, instead of the one you’re choosing to become.
The Quantified Selves paradigm is not for every organization or every part of the company. It’s designed to assess strategic decisions, professional judgment, long-horizon commitments, the cultivation of capability—how subjective decisions are made when man and machine are working together. Schrage argues that the ultimate purpose of agentic AI is not simply to automate decisions. It is to cultivate better decision-makers. Organizations that deliberately govern how agents shape human judgment may ultimately outperform those that optimize only for speed and efficiency.
Michael Schrage’s working paper “Who’s Ally Is ‘Your’ Agent,” is available in full on the IDE website.