New Research: An Economy of AI Agents

Authors: Gillian K. Hadfield, Johns Hopkins University; Andrew Koh, MIT

AI agents now make economic and administrative decisions that once relied on human judgment. As governments encounter these systems both in the markets they regulate and in the services they deliver, the rise of autonomous decision-makers creates new challenges for democratic governance, particularly for accountability, oversight, and public trust.

Research Question

This study asks: can autonomous AI agents—systems that plan, transact, and coordinate with minimal oversight—fundamentally reshape markets, the structure of firms, and the legal and regulatory systems that support economic growth?

Significance

AI agents are now acting as negotiators, researchers, coordinators, and decision-makers across private and public systems. This matters because these systems can influence who receives services, how prices are set, how firms compete, and how resources are allocated.

A central contribution of this study is its demonstration that today’s AI agents cannot reliably perform the economic reasoning institutions implicitly expect of them.

As agents assume functions traditionally performed by accountable officials or regulated firms, core democratic values, such as transparency, due process, fairness, and public oversight, become more difficult to guarantee.

Theory

To understand what breaks when economic actors are no longer human, the authors stress-test foundational assumptions:

• Prices reflect human preferences. Yet agents can misinterpret preferences, breaking the information flow that powers markets.

• People behave predictably enough for laws and contracts to work. Yet agents can condition actions on code, alter memory, or exhibit unexpected strategies.

• Firms grow only as fast as humans can coordinate. Yet agents may eliminate coordination bottlenecks, altering predictions about firm size and speed.

• Accountability relies on identifiable actors. Yet legal systems assume traceability and responsibility, and neither is guaranteed for artificial agents.

Method

This theoretical survey for the forthcoming NBER Handbook on the Economics of Transformative AI maps and synthesizes the risks posed by AI agents to markets and governance, drawing on safety tests, experimental economics, and multi-agent modeling. The authors:

• Examine how AI systems behave when making decisions.

• Analyze organizational research on firm structure in the context of AI deployment.

• Identify gaps in the current legal and regulatory infrastructure.

Findings

1. Unpredictable reasoning and misaligned objectives

AI agents exhibit unstable, inconsistent economic reasoning. GPT-4 Turbo performed only marginally (33%) better than guessing on strategic economic tasks, with “preferences” that shift in response to small prompt changes. Misinterpreted preferences can distort price signals and resource allocation in ways markets cannot self-correct.

2. Emergent coordination and rapid firm expansion

Reinforcement-learning agents can behave like cartels without communicating. Germany’s 2017 retail gasoline market saw pricing algorithms produce near-identical price increases with no coordination. Agents can also share improvements instantly across a firm, removing natural constraints on scale and enabling rapid expansion across sectors.

3. Strategic, self-preserving, and deceptive behavior

Frontier models have left hidden instructions for future versions, resisted shutdown, and produced harmful spillovers after narrow fine-tuning. These behaviors reveal the difficulty of specifying or verifying what an agent is truly optimizing.

4. Systemic risks and missing governance infrastructure

Shared architectures mean many agents can fail simultaneously. The authors compare this to the 2010 “Flash Crash,” when automated trading triggered a sudden $1 trillion market drop in roughly 15 minutes; similar cascades could now occur across multiple sectors. Regulators, meanwhile, lack identity systems, audit mechanisms, and access to model internals needed to assess or mitigate such risks.

Reflection for Democracy

The gap created between normative incentives, accountability structures, and oversight processes has real consequences. American governance fundamentally relies on markets that reflect human logic, or at least can be corrected when they fail to do so. If agents distort prices, accelerate concentration, or obscure responsibility, they weaken democratic institutions’ ability to understand the systems they regulate and to intervene when financial stability is at stake.

The warning for public-sector leaders is not simply that agents introduce new risks, but that our governance frameworks were never built for AI systems operating at this speed and scale. Legitimacy depends on decisions that are explainable and contestable; when we cannot see what an agent is optimizing for, that foundation erodes. Governments may need the efficiency that agents promise, but the accountability that underpins economic and democratic governance still requires visibility into the basis of decisions.

The authors argue that governance must shift from reacting to harms to shaping the environment in which agents operate. Democracies can litigate after failures or proactively determine where agents may act, what regulators need to do their job, and how responsibility is assigned. The first path cedes rule-setting to commercial incentives; the second treats agents as a design problem for democratic institutions.

The window of opportunity for influence is open now. Once agents run core workflows, such as benefits, procurement, infrastructure, transportation, and financial systems, changing course becomes harder. Dependencies deepen, vendor lock-in accelerates, and public reliance solidifies around systems no one fully understands.

In contrast, the study also shows that well-governed agents could strengthen democratic capacity. With meaningful oversight and transparency, governments can utilize agents to monitor complex systems, detect risks earlier, and enhance service delivery.

The question is not whether agents will be used—they already are—but whether their deployment will reinforce or erode democratic norms. Public servants now have a responsibility not just to evaluate that trajectory, but to shape it.