The very phrase “artificial intelligence” is an acknowledgement that AI is, in a pretty fundamental way, a likeness. It’s an imitation of human intelligence. One question which occasionally troubles us is how good this imitation could be. Could it ever get so perfectly alike that it turns out to be the same? Is an AI—could it ever be—the very same sort of thing you are? (I mean in principle. This isn’t a question about the economics of the current AI boom, or the stability of a generalized Moore’s Law. It’s a question about the very idea of a technology like AI.)
One reason why this question is interesting is that it’s one of those recalcitrant questions which divide people so completely that each side seems not to understand the other at all.
The AI enthusiast says, “Of course an artificial intelligence could be a real intelligence! Name anything that distinguishes you as a human being. Anything I can actually confirm, I mean. Like, I could ask and you could show me you can do it. Name anything, and an AI can do it too. Or if it can’t yet, there’s no reason why it couldn’t in the future. We used to think a computer couldn’t beat the best of us at chess, then at Jeopardy, now at translation, research, songwriting, and the list goes on. We just keep retreating. Don’t bet on the side that just keeps retreating.”
The AI detractor says, “You’re missing the point. It’s a technology, a tool—not a person, but a thing made of circuits and wires. We know because we made it. We can see, down to the circuitry, that all that’s happening here is that some electric charges are moving around in a pattern that we set up to give us certain outputs when we prod it in certain ways. The most you can say is that a human can use an AI to translate, research, or whatever else. The model isn’t doing those things by itself any more than a word processor is writing—or, for that matter, any more than the hammer is building the house.”
The enthusiast says, “But… particles moving around—that’s all you are too! Talk to literally any neuroscientist! And the complexity of a Large Language Model may rival that of a brain, even today.” The detractor says some version of “Not so!” And it goes on from there.
I want to think about what’s going on in this disagreement. First, it will illuminate some of what’s behind assertions such as that AI does not have (and could never have) a soul, or the that AI is not (and could never be) conscious, or what Pope Leo says in his recent encyclical, that AI is “a valuable tool that requires vigilance”—no more than a tool.[1] Some may read claims like this as simplistic assumptions—and, to be fair, the reasoning behind them is hardly ever spelled out—but there is actually some real argument behind them. Second, this investigation can tell us something about what AI is—even more deeply, about what we are, and what we’re doing.
Indeed, what follows may be considered a reflection on what it is to be an image of God, rather than an image of an image of God.
The enthusiast’s side is pretty straightforward. If I type into a chatbot textbox “please translate ‘granatappel’ into English,” what appears on the screen will be some words to the effect that “granatappel” means “pomegranate”. If one had asked a human that question, and if a human had said the same words that appeared in the chat window, that would have been some evidence that the human speaks Norwegian—or at least knows a few Norwegian words, or maybe just a few words in a nearby language like Dutch. So, says the enthusiast, it’s equally evidence that the chatbot knows English (or close enough).
This is more than just the sort of translation from input to output that digital technology always does. When a Word doc comes to have a poem on it, it translates keystrokes to digital text, but you wrote the poem. By contrast, when you ask ChatGPT a question, there’s no one behind the curtain pulling levers to produce that answer. People did (sort of) design the Large Language Model (“LLM”), but the LLM itself then produces each particular output on its own.
The defensive part of the enthusiast’s case is just a general parallel of the sort I hinted at already. The detractor will say, “But it doesn’t know anything. It’s just a big machine in a certain electronic state.” The enthusiast says, “And you’re just a smaller machine in a certain state. There’s no important difference here. At the end of the day, neurochemistry and electronics are both just the material for input-output functions.
I know how that sounds, but let’s not hide from the truth just because it’s unromantic.” There’s plenty the enthusiast can point to in support of this parallel.
There’s the idea that a piece of you, even of your brain, could be replaced with an electronic device if only it could interface properly with the native material.
There’s the idea that every mental event—a thought, a hope, a twinge of affection, etc.—has a physical correlate, some very complicated and no doubt elastic set of fibers firing off signals. The enthusiast will admit that these are not actually empirically confirmed principles—more like bets. But don’t they seem like safe bets?
And, at the end of the day, there’s also sheer intuition. If WALL-E rolled up to you, looking up at you with his birding-binocular eyes, and handed you a flower, you would read it as an offer of friendship. That is, we automatically, even unavoidably, read WALLE as a person. AI can reproduce these sorts of distinctively human gestures—or, at least, things we cannot help but read as such. We thank ChatGPT for its answers as if they were displays of goodwill. Myself included.
___
The detractor, meanwhile, will probably agree that we cannot help seeing WALL-E as a person. But still, she says—and even if you could replace any part of her you like with electronics without loss, and even if you could show that every mental event has a specifiable physical correlate—still, it doesn’t follow that an LLM knows anything, or makes plans, or sympathizes, or wants, or wonders… or (more controversially but for the very same reasons) that it can spell, or do math, or speak, or give advice, or infer, etc., etc.
For the detractor, it does not matter how the machine looks. Likewise, it doesn’t matter how a person looks—for it’s the very same line of thought which leads the detractor to say you cannot stop being a person by failing certain tests any more than you can become one by passing them. A robot could be extremely convincing, but it would not therefore be a person any more than an actress is Lady Macbeth. There is a difference here between the very logics of thought which apply to people and to machines. And that gap, says the detractor, is insurmountable.
Most detractors, of course, wouldn’t put it that way. But I do think it’s something like this line of thought which is very often behind their detractor’s stance. It’s subtle, so I think it will be useful (both to the enthusiasts and to the detractors themselves) to see it spelled out a little.
___
There are two pillars of the detractor’s thinking: one offensive, one defensive. Let’s call the offensive pillar ‘pre-personalism’. I don’t mean just personalism the intellectual movement—I mean what underlies it. I mean the sheer appreciation of what it is to be a person. It’s sort of thing we are expressing when we talk about how valuable people are, sometimes in the mode of Hallmark mysticism (think, as some friends share a bottle of wine at the end of a TV movie, “So he’s the one who made me realize… it doesn’t matter what you do, it’s who you’re with”), or actual mysticism (think St. Teresa of Avila or St. John Paul II; Khalil Gibran or Gerard Manley Hopkins; even Nietzsche or Camus), or just ordinary stuff-of-life facts like that murder is evil or that birth is miraculous.
This “offensive pillar” of the detractor’s position isn’t the sort of thing you can just put in propositional form and hand to someone on a piece of paper. It has to be shown. It can be shown, I think—it’s genuinely objectively verifiable by observation—just not in a sort of mathy, systematizable way. So we find ourselves in the unfortunate situation that, if the detractor wants to insist on it and the enthusiast wants to deny it, they’ve got very little to say to each other after that. The detractor is never going to be convinced that what she sees isn’t real. The enthusiast would have to change her way of thinking without antecedently seeing any reason to do that.
Bookmark that. Let’s talk for a moment about the ‘defensive pillar’. The defensive pillar is why the detractor is so resistant to (what the enthusiast thinks of as) counterevidence: why she will never concede that an AI is a person, or even “just as good”, no matter what the AI can do.
Stubborn as that sounds, the reason is actually just a pretty simple observation. Imitations of things—photographs, special effects, forged signatures, counterfeit coins, toupees, sugar-free cookies, etc., etc.—can be very good without therefore being the things they’re imitations of.
A forgery of Pete Rose’s signature, for example, is not Pete Rose’s signature. Not even if it’s an absolutely indistinguishably perfect replica of a signature he himself actually put on a baseball at some point. Signatures are not things you can define in terms of the shapes and weights of markings. If it wasn’t Pete Rose who signed it, then it’s not his signature. In fact, no set of characteristics you could test for (no matter the level of cleverness or detail) would be enough. An impossibly perfect sci-fi-type molecule-level duplicator, if you put a Pete-Rose-signed baseball in on one end, would still pop something out the other end that does not have Pete Rose’s actual signature on it. Only a copy.
The general point the detractor is trying to make here is that you can have a copy, even a copy as perfect as you could imagine, that’s still just a copy. It’s not the real thing, no matter how it looks, no matter what scientific tests it passes. The real thing is, we might say, irreplicable. You cannot make another of the real thing just by getting all the matter into the right arrangement. If you wanted a second genuine article, you’d have to do something more than just expertly arrange some matter. Ask Pete Rose to sign another ball, for example. And that is because the genuine article is not just a matter of having all the right material in the right arrangement. It’s something more than that.
(For readers who are curious enough to look further into it, this sort of point pops up every now and then in philosophy, where it gets far more development than here. John Searle’s Chinese Room thought experiment is perhaps the most famous example. His argument was very like the detractor’s argument here. One other example, a personal favorite and one highly relevant to the present subject, is Michael Thompson’s discussion of the swamp man in Life and Action.)
Now, you might be thinking this “irreplicability” must be a rare property, even a mysterious property, for things to have. Is it true of anything besides signatures?
It is! In fact, it’s not rare at all. Here’s a very brief list: soap, starlings, avocado toast, hedgehogs, hedgehog quills, hairbrushes, hammer dulcimers, the members of any badminton team, Neil DeGrasse Tyson, Oliver Twist, Oliver Twist, the letter ‘b’, every word spelled with the letter ‘b’, E. coli, Belgian techno-anthem “Pump up the Jam”, the possible world in which I am a Belgian techno artist, the savannah, my desk, laughter, joy, Saturdays, peanut butter.
In general, anything that’s alive or a part of something that’s alive, anything of a species, anything with a meaning, anything with an intrinsic purpose—artifact, action, abstractum, whatever—any possession (as such), any member of any group (as such), anything the nature of which consists in some relation to something else… all of these are irreplicable. Depending on how you count, that might be almost everything.
The question now is whether people are like that too.
Human beings are certainly irreplicable, because ‘human being’ is a designation for a living thing, a member of a species. You cannot be a member of a species without being related in the right way to the life of the species. As a case in point, “convergent evolution” could not make two kinds of bird the same kind of bird just by making them phenotypically (even genotypically!) identical.
‘Person’, though, is a different sort of term. In philosophical English, it applies at least in principle to things other than humans: angels or Martians, for example. And there’s dispute about what makes something a person. Most people say it comes down to some characteristic. Typical candidates include reason, self-consciousness, or selfreflectiveness.
I myself like the ‘reason’ option, but honestly, that’s an idea that needs just as much unpacking as person did in the first place. And—here’s the rub—a rational thing is an irreplicable thing, in our sense from above. In me, solving a math problem is an exercise of reason. In a calculator, solving a math problem (if “solving” is even the word) is just a more complicated and less overtly visible version of what’s going on in an abacus. It’s not an exercise of reason. It doesn’t matter if the calculator gets more complicated and does more math and starts calling itself a “computer”. Rationality, in short, is a (capacity for a) form of activity, and processes which look a lot like that activity do not automatically count as rationality just because of that appearance.
Now, ever since my list of irreplicable things a few paragraphs up, the enthusiast will have been pulling her hair out: “No, being a person is simply a matter of [insert fully empirically specifiable characteristic here]!”; “Soap!? Soap is not irreplicable!”. As much as I do think it would be worth our time to continue the dialogue down that path, and though I do think the detractor has good arguments for all the above, I am going to resist. I do not want this to become an extremely long technical paper. Right now, all I’m trying to do is explain how detractors (like me) are often thinking about the issue. This is why they will insist that an AI is not a person no matter what the AI “accomplishes”. They are thinking of persons—rationality, intelligence, whatever else being a person might consist in—as irreplicable things. And if they’re right, then it doesn’t matter how impressive, how groundbreaking, how bafflingly perfect the copy is—it can only ever be a copy. It doesn’t matter how on-the-dot the forgery of Pete Rose’s signature is.
The moment you see it this way, you’ll see other things this way too—that is, not just the difference between people and their imitations, but also the differences between things people do and their imitations. There’s a kind of irreplicability that shows up in our actions too. I’ll give three examples: art, education, and philosophy.
___
Suppose a poet goes for a walk, sees a snail floating along the garden border, sees a hedgehog turn away, and feels a twinge of sadness. When he returns to his room, he has an idea, and he sets this idea down as a poem. “The snail moves like a / Hovercraft…”. And the hedgehog who turned away—why? “We wonder what a hedgehog / Has to hide, why it so distrusts.” Perhaps the hedgehog is just too good for us. Not even ironically—maybe it just has a certain purity or innocence or something to it, and does not negotiate with the defiled or the defiler: “We forget that never again / Will a god trust in the world.”
(This is Paul Muldoon’s “Hedgehog”. I just took Poetry Foundation’s current poem of the day as I wrote this for my example.)
What is happening here, as the poet writes? Well, at the most braindead-simple level, the poet is saying something. He is taking something he sees and giving it to us, his readers, in the medium of nicely arranged words. Here, then, is another way to put that thought: a poem is a gift, the substance and vessel of a thought which is given by the poet to the reader.
Let’s contrast this with a game I might play with my dog. I hold up a treat, and my dog goes to a giant pile of flashcards and searches through it for a little set of cards to bring me. The dog doesn’t know what words are on the flashcards, obviously, because dogs don’t know words. But the flashcards are also marked with colors and patterns (which I have cleverly coupled with words ahead of time), and the dog does know, after playing this game a million times, what sorts of colors and patterns he can bring me if he wants an enthusiastic pat on the head (or perhaps even a second treat!). One day, I hold up a treat, and the dog brings me the cards: “the” “snail” “moves” “like” “a”, and “hovercraft”.
Obviously what the poet is doing is nothing like what the dog is doing. The poet is speaking, expressing an idea by using words. The dog is not even using the cards as words— to the dog, they’re just messes of marks and colors. He’s not a literate dog—only an extremely well-trained dog, who, with practice, can recognize which sorts of combinations of cards will get him treats.
Now, is an AI more like the poet or more like the dog? The detractor says, “The dog, of course! Capable of more reliably producing what we read as sentences than the dog is, I’m sure, but it’s doing the same kind of thing. The dog isn’t treating these as words, but as to-be-given outputs.[2] The AI is doing the same.”
For the detractor, there is no thought here to be given—nothing which was put into poetic words, but only an algorithmic procedure which caused those words to appear. Nor is there anyone to give the word-arrangement, if the AI is not a person. Nor is the AI even using words as words—i.e., saying anything—any more than my dog.
An AI can certainly output something which users will interpret as a poem. And maybe at that point, upon being received, it is a poem. That doesn’t mean the AI wrote a poem. No more than the universe “drew” Orion.
___
Everyone reading this has presumably “had an education”, or at least an attempt at one. What does that mean? What have you had if you’ve had an education?
As with art, your education consisted at least in some “material” or “observable” things, so to speak. Certain information was not in your head, then it was. Certain procedures were beyond you—spelling ‘horse’, painting a picture of a horse, citing your sources, solving a calculus problem—and later they weren’t. Yes, education surely consists at least in lots and lots of these sorts of discrete, testable things. Impersonal things, even: teacher A could have told you that ‘horse’ has an ‘e’ at the end just as well as teacher B— and, at some level, the thing you have once you’ve been told, the knowledge that ‘horse’ has an ‘e’ at the end, is exactly the same no matter who told you.
Now, I’m not saying that’s an easy and uncomplicated thing. I cannot just tell you that what simplifying “(d/dx)(x³ + 2x² + 4)” demands is that you move the little numbers down to the fronts of the ‘x’es, multiplying if necessary, subtract 1 from each little number, and get rid of that ‘4’. If you’ve never seen a derivative before, you’ll have no idea what I’m telling you to do. A good teacher will have a way to explain what’s going on here which actually helps the student see. She will tune it to the student herself, seeking out and targeting the student’s remaining misunderstandings. Teaching is a craft.
All the same, even that is not all a teacher is. Teachers aren’t just experts at getting facts and skills quickly and permanently into their students. I’m not even saying a teacher’s real job is to give students “the whole picture”. More than that! A teacher is, fundamentally, a person who is (usually on some particular dimension) what the student is trying to be. The math teacher is proficient in mathematics; the math student is trying to be the same. The literature teacher is a good writer of stories; the literature student is trying to be the same. Not exactly the same, of course, but the version of herself who is a good writer of stories, or proficient in mathematics, just as the teacher is. She is being initiated into the dimension of human understanding which the teacher represents.
Teachers, in other words, are exemplars. What they are trying to give their students are not disembodied abstracta which they divvy out like payroll managers. What they are trying to give their students are little (usually subject-specific) bits of themselves. A teacher is not like a delivery person, having nothing especially to do with either sender or recipient, nor with the package itself. A teacher is more like a miniature parent, whose children, though they will inevitably be their own people and go in their own directions, will always be running with their mothers’ feet and building with their fathers’ hands.
In short, to be an educated adult is to be a collage of all who have formed you. We do not make ourselves—we are not even made of only ourselves.
An AI—or a textbook, for that matter—can give you facts and guide you in developing skills. But an AI is not the sort of thing you are trying to be. And so, says the detractor, an AI cannot educate you. It just doesn’t matter, it’s totally irrelevant, whether an ‘AI instructor’ produces better test scores than a teacher, or gets better student evals, or even genuinely and verifiably causes there to be more information in the students’ heads (shocked as I would be if it turned out that way).
If we ever did try to replace teachers with LLMs, I’d bet my hippocampus there would be patently obvious, maybe even empirically verifiable, losses. But the point is that, even if there weren’t, it wouldn’t matter. Education would not be occurring. What we would have would just be a bunch of children finding their own way in an extremely well-fenced-in environment constructed by some distant committee.
___
Let’s again begin by asking a “what is it?” question. What is philosophy? As any philosopher will be the first and the second to tell you, there’s disagreement on that. But, again, all I’m trying to do at the moment is explain the detractor’s point of view. So here is one answer the detractor might give.
The world is really complicated. You may think you have a pretty good sense of how things are, but if you press on that understanding—literally anywhere, even with just a finger—light will start coming through the cracks. You know what a sheep is, for example. Well—do you? Tell me what a sheep is, then. The very first things you reach for will probably be stuff like the number of its legs or the material it’s coated in. But you know perfectly well that cutting off its legs and shaving off its wool would not make it no longer a sheep.
You’ll want to take another shot, I’m sure. But I, being at the far side of an alreadywritten-down bunch of words, will not be able to play this game much longer. So I hope you’ll take my word for it that the game can be played much longer. Just read some Plato. It is really, really hard to articulate accurately what anything is. And that’s just us pressing on ‘sheep’. Do you know what a sheep is? What is knowing something? And how do you know things? Or: what a sheep is. What is it to be something—to have an identity or essence? What is a sheep made of? (Atoms? But which ones? Surely not the ones making up the grass the sheep is chewing. What about its gut bacteria? What about its sperm or eggs?) How does the sheep run without running straight into a paradox of motion? And so on, and so on. The world is really complicated. Press anywhere.
I’m not saying you don’t know anything. You obviously do. I’m saying that real understanding is the task of a lifetime. Indeed, it’s the task of countless lifetimes. Your own understanding is the fruit of an enormous collective effort on the part of every worldunderstander who ever lived and passed anything on down the line of world-understanders to you. And you, too, will pass on large parts of your understanding to your children, and small parts of your understanding to the people you pass along your way, and they will do the same, on and on forever.
So, about this giant collective project: is it something an AI is, or could be, part of, the way you and I are? “No,” says the detractor. Because, for that, the AI would need to be capable of forming an understanding of things. That’s the whole game, after all. But what an AI does, at the very very most, is to serve as a dynamic and extremely complicated repository for things human beings have understood. Its “training data”, we call this. But that’s an odd term, because the AI did not learn information—it had its outputs tuned by a selection process so as to systematically reproduce bits of information which humans put on the internet at some point. All its “knowledge” is just a tendency to paraphrase things someone—some person—wrote; everything it “says” is plagiarism.
This is true even of the more “creative” outputs which aren’t just presentations of bits of information. ChatGPT’s indominable moral relativism, for example, is just a response pattern which is both common in its “training data” and considered safe and inoffensive by its tuners. When it refuses to tell you whether abortion is wrong and asks what you think instead, that, too, is just a pattern it reproduces as output because it was selected for in its formation.
In short, then, an AI is a bit like a book: a procedurally generated choose-your-own-ending encyclopedic novel with a thousand editors and several billion coauthors. It does not have any understanding which it can share with you, because it does not know or understand things, and does not say things. (Think again of my dog.) Its outputs may sometimes be helpful to you, even edifying. Some text in an output bubble may genuinely help you think through the meaning of your life. That does not mean the AI is doing philosophy. It is not a participant in our collective attempt to understand things any more than a book (as a book). Its authors may be participating, but the LLM is not.
____
What we think AI is (or can be) depends on what we think we are. What we think AI can do depends on what we think we have been doing this whole time. If we have just been interesting machines all along, then AI is the same general sort of thing we are. If we have just been arranging markings or transferring information, then AI is doing what we are doing. In that case, what’s stopping it from making art, educating students, participating in our science and philosophy… having friends, having rights, getting sued, etc., etc.? But if we are people, and if we have been making art, teaching, and understanding—and if there is (as the detractor says) more to all of that than its material—then AI is not what we are, and it is not doing what we are doing. There are categorical differences between people, art, etc. and any arrangements of particles or of movements of particles.
Your views on AI are your views on yourself. Consequently, the detractor will say, if you are thinking of replacing artists and teachers and thinkers with AI, then you must not know what art and education and philosophy are in the first place.
Does any of this matter? A cynical sort of enthusiast might point out that, all this time, I have been arguing that an AI is not a person, and cannot make art, or teach, or do philosophy, even if there’s no way to tell the difference from a purely material perspective between what the AI is doing and what a person would do. “Well,” says the cynical enthusiast, “if there’s no difference, then there’s no difference, right? Why try to avoid something if it makes no difference?”
Again, it’s “no scientifically testable difference”. The assumption that that’s the only sort of difference is the entire issue. Besides, again, it would be shocking to me if just replacing all these things with AI imitations didn’t make a scientifically testable difference. My point has just been that, even if it didn’t, we would still have lost something.
What would we have lost? Here we return to the first pillar of the detractor’s case, which I very briefly noted above: the detractor’s mystical sense of the incredible value of humanity, of personhood. For that’s what we’d be losing, of course. (Not the value, which is immutable, but the sense of it.) I would think it’s excruciatingly obvious that that’s a real loss. But if the cynical enthusiast disagrees, I don’t know how to convince her. My only recommendation is that she go fall in love with someone.
____________
[1] Magnifica Humanitas, §100. I should also probably just come out and say that I know this piece has lots of em-dashes in it. No, I did not use AI to write it. I just love em-dashes.
[2] In truth, I don’t think an AI is even doing that. But let’s not overcomplicate the point.
