The Illusion of Human Control: AI Agency and Adaptive Behavior in Collaborative Knowledge Work
COM2 Level 2
MR3, COM2-02-26
Abstract:
As artificial intelligence systems evolve from reactive tools into autonomous agents capable of independent planning and execution, a fundamental tension emerges: does greater AI agency also mean diminished human control? This thesis examines this control-performance dilemma through two essays, distinguishing between two related but distinct forms of control that are exercised in human-AI collaboration.
Essay I examines how varying levels of AI agency influence the extent to which human collaborators shape and assume responsibility for jointly produced work. Using a field experiment embedded in a recruitment process (N = 358; agentic AI, generative AI, and no-AI conditions), this essay shows that higher AI agency is associated with more abstract communication and reduced behavioral engagement from human collaborators, which together increase output homogeneity while diminishing collaborators' sense of responsibility for the outcome. These effects are further shaped by individuals' interaction norm about how actively humans should remain involved in AI-assisted work, revealing a boundary condition under which agentic AI's performance advantages hold.
Essay II investigates whether, and under what conditions, humans can redirect AI behavior through the context they provide. Using a 3×3 factorial design spanning knowledge structure and conflict-expression type across 4,500 LLM interactions, this essay demonstrates that large language models' responsiveness to conflicting context is highly conditional. Models readily detect the explicit entity-level conflicts and adapt more to semantic conflicts. However, their attention to pragmatic conflicts frequently fails to surface in their stated reasoning, leaving this failure undetected. A proposed "Leaky Pipeline" model characterizes where in the reasoning process this epistemic control breaks down, with direct implications for organizations that rely on supplying updated context to govern AI behavior.
Together, the two essays present a unified account. As AI agency increases, human control does not disappear, but becomes less direct, shifting from direct engagement and execution to framing, checking, and oversight.