They are so happy

SANDY: Not to this mechanist. The way I see it, consciousness has got to come from a precise pattern of organization - one that we haven't yet figured out how to describe in any detailed way. But i believe we will gradually come to understand it. In my view consciousness requires a certain way of mirroring the external universe internally, and the ability to respond to that external reality on the basis of the internally represented model. And then in addition, what's really crucial for a conscious machine is that it should incorporate a well-developed and flexible self-model. And it's there that all existent programs, including the best chess-playing ones, fall down.

CHRIS: Don't chess programs look ahead and say to themselves as they're figuring out their next move, "If you move here, then I'll go there, and then if you go this way, I could go that way ..."? Isn't that a sort of self-model?

SANDY: Not really. Or, if you want, it's an extremely limited one. It's an understanding of self only in the narrowest sense. For instance, a chess-playing program has no concept of why it is playing chess, or the fact that it is a program, or is in a computer, or has a human opponent. It has no ideas about what winning or losing are, or -

PAT: How do you know it has no such sense? How can you presume to say what a chess program feels or knows?

SANDY: Oh, come on! We all know that certain things don't feel anything or know anything. A thrown stone doesn't know anything about parabolas, and a whirling fan doesn't know anything about air. It's true I can't prove those statements, but here we are verging on questions of faith.

PAT: That reminds me of a Taoist story i read. It goes something like this. Two sages were standing on a bridge over a stream. One said to the other, "I wish i were a fish. They are so happy!" The second replied, "How do you know whether fish are happy or not? You're not a fish." The first said, "But you're not me, so how do you know whether I know how fish feel?"

PARTICIPANTS: Chris, a physics student; Pat, a biology student; and Sandy, a philosophy student.

~ Selection from "Metamagical Themas: A coffeehouse conversation on the Turing test to determine if a machine can think." Scientific American, May 1981, pp. 15-36.