A Different Perspective on Searle’s Chinese Room

Consciousness is a difficult problem to grapple with, mainly due to its inherently subjective nature. We don’t yet understand the brain well enough to say much about how it does anything, let alone to describe how it gives rise to (or doesn’t give rise to) conscious experience. Due to this limitation, we’re currently reliant on the realm of philosophy and its thought experiments to provide insight into the world of consciousness. These arguments can be powerful, but their indirect nature leads to some “looseness”, with their veracity lying more in the inclinations and interpretations of the reader than in any external truth. However, as they’re all we have (at least until significantly more progress is made in neuroscience), it’s worth pushing the ideas as far as possible.

One famous argument in this space related to machines and consciousness is Searle’s Chinese Room, which he summarized as follows (the full version can be found here):

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a database) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

While Searle does not directly refer to consciousness in that formulation, the ideas of understanding and consciousness do seem to be closely linked, with Searle himself drawing the following conclusions from his argument years later:

I demonstrated years ago with the so-called Chinese Room Argument that the implementation of the computer program is not by itself sufficient for consciousness or intentionality. Computation is defined purely formally or syntactically, whereas minds have actual mental or semantic contents, and we cannot get from syntactical to the semantic just by having the syntactical operations and nothing else. 

Searle’s argument certainly feels compelling on first read, and I will not attempt to directly refute it in this post (most of the standard replies can be found here). Instead, I’ll lay out a similar (though more lengthy) thought experiment which suggests the opposite conclusion, and will leave it to you to decide which feels more compelling.

SHRDLU was an interesting computer program put together by Terry Winograd in the late 1960’s. It was designed to operate in a world of blocks (see the image below) where the user could enter instructions asking SHRDLU to take action in the domain (for example, “pick up the green pyramid and move it onto a green box”). The simple domain allowed for better performance, and SHRDLU was fairly effective at understanding its environment and properly moving around the blocks, though it relied on relatively simple language processing techniques. For the sake of example, we can treat SHRDLU as being hard-coded and inflexible, with specific actions paired up to specific input sequences ahead of time by the programmer (much like the instructions Searle would follow in the Chinese Room).

However, we can also imagine a more complex version of SHRDLU, say SHRDLU 2.0, which is instantiated as a robot within this simple domain of blocks. SHRDLU 2.0 has visual and tactile sensors, and is able to learn by itself about how to form concepts within its limited domain, perhaps through a type of neural network approach. The programmer would not specify anything like “block” or “green” inside SHRDLU 2.0, but would instead simply structure the underlying algorithms in a way which allowed for learning, and then let SHRDLU 2.0 explore its world. These algorithms would function to make sense of the regularities of the environment, leading to SHRDLU 2.0 developing “concepts” such as “block” and “green” (and perhaps with something like reinforcement learning being used to encourage correct action based on user requests). Taking a biological perspective, we might say that creatures like worms and bugs experience the world in a similar way to SHRDLU 2.0 – they’re able to function in and learn things about their narrow domain (things like what concentration gradients suggest the presence of food, etc.), though they’re far from having any “true” understanding or consciousness.

Taking this idea further, we can imagine a robot similar to SHRDLU 2.0, but without the domain limitations, and with correspondingly more robust learning algorithms in place (powerful enough to derive regularities from this significantly more complex environment). Let’s call this new machine SHRDLU 3.0 (and please note that this type of machine is far beyond what we’re currently able to construct, so try not to limit your visualization of it based on the current state). SHRDLU 3.0 would be effective in forming concepts based on its inputs to guide its actions in the world – for instance, it might form a general “object” concept, which would capture the fact that matter seems to group together in certain ways in the world – and then from this object concept it might identify things like inanimate objects (which don’t move unless moved), regular animate objects (things like water flowing, where there’s movement but it follows a pattern), and highly animate objects (things like animals, that don’t follow a simple set of rules). I’m using these concepts as examples, but it’s important to note that SHRDLU 3.0 would be forming these concepts by itself (based on the actions of its internal algorithms) and would not be “given them” directly by any programmers (so we couldn’t say ahead of time which concepts would actually form). So far, SHRDLU 3.0 may appear only to be a more complex version of SHRDLU 2.0, simply with a broader variety and depth of understanding. However, something interesting happens when we make the jump to SHRDLU 3.0. As its domain is not limited, the domain which its algorithms learn from happens to include itself. This means that, in the same way that SHRDLU 3.0 is able to identify the regularities of its world like objects and inanimate objects, it is also able to identify that there’s a particular always-present object (SHRDLU 3.0) in its domain – and that this object seems to be the “do-er” of the actions SHRDLU 3.0 makes. The hierarchical, associative web of concepts which SHRDLU 3.0 forms of the world and uses to guide its actions is not only much deeper and broader than that of SHRDLU 2.0, but also of a different nature, due to the fact that it includes itself (the hierarchical, associative web-making machine). SHRDLU 3.0 is obviously an incredibly complex machine, so at least at first, its internal pattern recognizer won’t have a very robust pattern for it – again tying it back to biology, you could imagine this rudimentary SHRDLU 3.0 as having an understanding of its world and itself similar to that of a mouse (or a cat, or a dog, depending on how robust you’re picturing the pattern recognizing abilities). However, if you imagine this program improving over time (for the sake of argument, we can assume there’s a whole bunch of SHRDLU 3.0s, which all vary slightly and reproduce with inheritability, and that there are limited resources on the world they inhabit, and the increased ability to recognize and incorporate the regularities of the world has survival and reproductive benefits…), you could see how SHRDLU 3.0’s could over time acquire increasingly powerful abilities to make sense of the world they inhabit, and of their own actions within that world. At a certain point, their ability to recognize the patterns of others of their kind might get so good (especially if they evolved to be social creatures) that they’d be able to form an abstract system for labeling and referring to certain concepts (language) – we can label this final version SHRDLU 4.0.

This turned into a bit of a long journey, but at the end of it, we’re left with SHRDLU 4.0 – an algorithm which uses language to talk to other instantiations of algorithms, which refers to itself in the first person, and which, if you asked it (in its own SHRDLU 4.0 language) whether it has consciousness would answer resoundingly in the affirmative. It would have a name for itself, and it might even be interested in philosophy, wondering about questions like “how did I get here?”, “why am I conscious?”, and “could a machine ever be conscious?”. From the outside, we wouldn’t be able to tell whether there was consciousness in there or not (just as we can’t tell for other humans) – but I would believe SHRDLU 4.0, wouldn’t you?