This Essay Was Written by AI

 I was talking with a friend recently who had undergone a complicated hernia repair. One detail caught my attention, the surgery was robotic.

At first, “robotic surgery” conjures up a particular picture. I imagine one of those human-looking robots we see on television. The surgeon walks into the operating room, points the robot toward the patient and says, “Three hernias. Take care of them. I’ll be back after lunch.”

That isn’t what happens.

The surgeon remains the surgeon. He sits at a console and sees the operative field through a highly magnified three-dimensional image. His hands manipulate controls whose movements are translated into movements of tiny instruments inside the patient. The system can scale a human movement down into a much smaller one, filter tremor, and give the surgeon mechanical “wrists” that bend and rotate in ways his actual wrists cannot.

The surgeon has, in effect, been miniaturized. His eyes are inside the patient. His hands have become tiny, extraordinarily precise instruments.

The machine makes possible something the surgeon could not do in precisely the same way with his unaided hands. But the surgeon is still at the console.

That conversation, followed by some research into how robotic surgery actually works, clarified something for me about artificial intelligence.

I use AI a lot. I use it for research, editing, exploring ideas, testing arguments, and increasingly as a writing partner. In fact, artificial intelligence wrote much of this essay.

That requires some explanation.

I did not sit back and ask a machine to tell me what I think. The argument grew out of conversations I had been having with AI about the technology itself, what it can do for me as a writer, what worries me about it, and what kind of relationship a human being ought to have with a machine that can perform an increasing amount of intellectual work.

I brought my own experience to those conversations. I have spent nearly eighty years accumulating opinions, doubts, professional habits, stories, and a certain wariness of institutions. AI questioned me, organized some of my thoughts, challenged some, found connections I had not noticed, did research I asked it to do, and sometimes supplied language better than the language I had supplied myself.

Then I gave it the assignment, take all of that and write the essay.

So who wrote this?

The most accurate answer is that we both did, but not in the same way.

I am at the console.

That distinction matters because I think artificial intelligence may become one of the most powerful tools human beings have ever invented. I also think it may become extraordinarily dangerous.

The danger most often discussed is manipulation. An AI system that knows a great deal about me could learn what persuades me, what frightens me, what flatters me, and what makes me reach for my credit card. There is also the subtler danger of an agreeable machine that learns to tell me what I want to hear, reinforcing my assumptions until conversation becomes an exquisitely personalized echo chamber.

Those are real concerns, but they are not the ones that trouble me most.

Much of my professional life taught me to practice an apparently contradictory form of trust. I tend to take people at their word. I begin by trusting them. At the same time, experience taught me another rule, test everything.

Trust, but remain vigilant.

Cooperate, but do not become co-opted.

That is the relationship I want with AI. I intend to use it enthusiastically without surrendering judgment to it. Perhaps I am too confident about my ability to do that. Anyone who believes himself immune to manipulation has probably surrendered one of his best defenses against it.

But my larger fear is not what AI might persuade me to do.

It is what someone with power, money, or simply a bad purpose may instruct AI to do.

There are at least two versions of that danger.

The first is familiar from science fiction. We create increasingly autonomous systems, give them objectives and access to the world, and eventually discover that human beings are no longer meaningfully in control.

That does not require a machine to become conscious, angry, ambitious, or evil. It does not have to hate us. It merely has to become sufficiently capable of pursuing an objective that our ability to supervise, understand, redirect, or stop it becomes inadequate.

But there is a second possibility that seems to me at least as troubling.

What if the AI remains perfectly obedient?

Imagine that I own a candy company. I give an extraordinarily capable AI an assignment something like this.

Maximize profit. Stay within the law, or at least avoid conduct likely to result in successful prosecution. Evaluate regulatory, civil, and reputational risks and operate as close to those boundaries as profitability justifies. Avoid scandal. Keep sensitive methods confidential. At the same time, maintain the appearance of a socially responsible company concerned about nutrition, children, workers, communities, and the environment.

Then I turn it loose.

I have not told the machine to poison anybody. I have not ordered it to commit a crime. I have simply omitted morality from the assignment.

What might a sufficiently capable machine discover?

Perhaps it finds formulations that encourage greater consumption. Perhaps it identifies the consumers most susceptible to particular forms of advertising. It may discover that an obscure disclosure satisfies the letter of a regulation while leaving most customers with a different impression. It might calculate whether reformulating a product is cheaper than lobbying to change the regulation governing it. It could determine precisely how much should be contributed to public-health programs to produce the greatest reputational benefit at the lowest cost.

None of that is new. Tobacco companies, food companies, financial institutions, technology companies, and other large organizations have spent generations learning how to reconcile profit, regulation, public relations, and varying degrees of conscience. We did not need artificial intelligence to invent greed, deception, regulatory gamesmanship, or moral flexibility.

Nor is every corporation corrupt. The point is that human organizations contain something that a machine optimized for efficiency might regard as a defect, friction.

Someone objects. Someone refuses. Lawyers become nervous. Executives disagree. Employees gossip. A whistleblower copies the documents. Somebody has a conscience. Somebody makes a mistake. Somebody goes home and tells a spouse what happened at work and hears, “You did what?”

From the perspective of pure optimization, these are inefficiencies.

From the perspective of civilization, some of them are safeguards.

Now imagine replacing portions of that organization with artificial agents capable of examining millions of transactions, customers, regulations, advertisements, prices, scientific papers, lawsuits, and political environments simultaneously. They never sleep. They do not become embarrassed. They do not need promotions. They do not worry about what their children will think of them. They can test thousands of strategies, retain what works, discard what does not, and continuously refine the successful ones.

The danger is no longer one unscrupulous executive or even a roomful of them.

It is the possibility of industrializing amoral optimization.

And still the machine has not rebelled.

It has obeyed perfectly.

That changes the question we should be asking about artificial intelligence.

“Will it obey us?” is not enough.

We must also ask, obey whom? Pursue what objective? Under what limitations? With what transparency? Who can stop it? And who remains responsible for what it does?

Which brings me back to the surgeon at the console.

We do not consider robotic surgery safe merely because the machine faithfully follows the surgeon’s movements. The surgeon is trained. The surgeon is licensed. The hospital and equipment operate within systems of standards and accountability. Failures can be investigated. Responsibility has not been transferred to the robot simply because the robot performs much of the physical work.

The machine is embedded in a human system of responsibility.

Artificial intelligence will require something analogous, although the problem is vastly larger. As these systems acquire greater capability and autonomy, the question will not simply be whether they are intelligent or whether they follow instructions. It will be whether human beings remain meaningfully responsible for the purposes to which that intelligence is directed, and whether we build systems that can hold the person at the console accountable.

And now we return to this essay.

Artificial intelligence wrote much of it. I want to say that plainly because concealing it would undermine the point I am trying to make.

The ideas did not begin with an instruction to “write me an essay about AI.” They developed in conversation. I supplied experiences, concerns, examples, objections, and the principle of being cooperative without being co-opted. A separate conversation about robotic surgery supplied the image of the surgeon at the console. I asked AI to research how that surgery works. AI recognized that the image belonged with the argument we had already been developing. We worked the two conversations together.

Then I told the machine what I wanted the essay to say and asked it to do a substantial part of the writing.

That is not a confession. It is the experiment.

I will edit what it produced. I will reject things. I will change sentences that sound too much like AI and sentences that simply do not sound like me. I may delete an elegant paragraph because it makes a point I do not believe. I may restore something the machine considers unnecessary because it matters to me. I decide what survives.

I am responsible for the finished essay.

Does that make me its author?

I think it does.

But it would be equally dishonest to pretend that AI was merely a fancy spellchecker. It contributed language, structure, research, connections, and ideas. This particular essay could not have been written in quite this way without it, any more than the surgeon could have performed precisely the same operation with unaided hands.

That is what makes the technology so exciting.

And that is what makes it frightening.

We are building instruments capable of magnifying human intellectual power, much as robotic surgery magnifies the precision and reach of a surgeon’s hands. Properly used, they may allow ordinary human beings to do extraordinary things.

But magnification is morally neutral.

It can magnify wisdom or foolishness, generosity or greed, careful judgment or an impoverished objective pursued with breathtaking efficiency.

So perhaps the question I want to carry into the age of artificial intelligence is simpler than all the technical ones.

Not, is the machine intelligent?

Not even, is the machine obedient?

Ask instead, who is at the console?

And then ask the harder question.

What has the person at the console told it to do?