If you’ve seen my YouTube channel and this blog, you’ll know that for the last few years—ever since ChatGPT first came out, really—I have written and spoken almost exclusively about the latest developments in agents and how to build systems around LLMs. But, for a while, I want to pivot away from that.

I want to go deeper: deeper into the past, and deeper into the fundamental questions involved.

The central question—and the deepest angst—of the AI age is this: What will humans do if part or all of their work is automated? At a more technical level, how should we design and implement that automation? These, however, are age-old questions—ones I’ve frequently covered on the blog, years before LLMs. All this has happened before, and all this will happen again.

AI agents have brought the automation of knowledge work to a much larger audience. Suddenly everyone is talking about agents that write code, operate software, conduct research, and carry out business processes. And almost every conversation eventually reaches for the same reassuring phrase: human in the loop.

Usually this means that a person will approve the agent’s work, verify it, correct it, or step in when something goes wrong.

But it is a cop-out. An all-too-easy catchphrase.

What exactly is the person approving? The final result? The path the agent took? Its interpretation of the goal? The evidence it chose to trust? What happens to the person’s skill when the agent performs the task a hundred times and the person merely clicks a button? If the agent fails on the hundred-and-first attempt, is the human still equipped to rescue it?

There is a deeper literature here

The fields of human factors, cognitive psychology, human-computer interaction, and automation have studied versions of these problems for decades. Researchers asked what happens when work is divided between people and machines. They studied how automation changes human skill, attention, judgment, and responsibility. They looked at how people recover when automated systems fail and how interfaces either support or sabotage that recovery.

Unfortunately, the present-day AI community has largely ignored this literature. We talk as though the arrival of language models created the problem of combining human and machine intelligence. It did not. Language models have given the problem a new form and a much larger surface area.

This older literature gives us something we badly need: a more precise vocabulary and richer frameworks for thinking about the joint system formed by a human, an agent, its tools, its interface, and the organization around it.

I want to follow several connected questions through this literature:

  • What are the different levels of automation, and how should authority be divided between a person and a machine?
  • How does someone maintain situation awareness when they are no longer directly performing the task?
  • When should a person rely on an automated system, and what does appropriately calibrated trust look like?
  • How do we preserve skill and recovery capacity instead of optimizing only for the happy path?
  • What makes a handoff from machine to human possible when the system is moving quickly and the case is already abnormal?
  • How should interfaces represent the signals, signs, and symbols people need to understand what the system is doing?
  • How do we design the entire human-machine system instead of optimizing one component?

That is the motivation for my new video series, Beyond Human-in-the-Loop. I want to go back to landmark papers in this area, understand what they actually say, and explain them. I intend to keep discussion of AI and agents to a minimum and focus on the works themselves. I hope people will naturally see the applications to the present.