Infinitely Malleable

This post contains spoilers for “1984” by George Orwell

George Orwell’s classic 1984 pressures readers to think deeply on numerous topics, including (but certainly not limited to) the definition of reality, the role of government, and human nature. I found his examinations of human nature to be especially interesting, as in the first half we watch Winston (the protagonist) rebel against the tyrannical government (“Big Brother”), and in the second half we watch this same government pick Winston apart (i.e. torture him) and put him back together as a new man, aligned both internally and externally with Big Brother. During this second half, one quote from Winston’s torturer, O’Brian, especially stood out:

O’Brian: We control life, Winston, at all its levels. You are imagining that there is something called human nature which will be outraged by what we do and will turn against us. But we create human nature. Men are infinitely malleable. Or perhaps you have returned to your old idea that the proletarians or the slaves will arise and overthrow us. Put it out of your mind. They are helpless, like the animals. Humanity is the Party. The others are outside – irrelevant.

“Men are infinitely malleable”. A strong claim, and one which we’ll spend the rest of this post unpacking. In the novel, we see it to hold true; Winston converts, with the closing line of the book stating of Winston, “He loved Big Brother”. But how true is it in actuality? This question resembles the classic “Nature vs. Nurture” debate, with the difference being that here we have an intelligent agent (e.g. another human, “Big Brother”, etc.) trying to mold someone a particular way. Let’s  look at this question from a couple different perspectives, starting from the abstract and working towards more tangible views. 

To start, it may be helpful to take a step back and look at what it means for any intelligent agent to be “malleable”. Broadly speaking, it seems there are two ways to shape an intelligent agent. The first is the same way we shape other inanimate objects; by simply rearranging the pieces. For example, you could imagine a neuroscientist shaping someone by going into their brain and changing up neural connections. It doesn’t seem that O’Brien had this method in mind when speaking with Winston, but it does feel important to call out – though we’re quite far from having the capabilities and knowledge to do this on the human brain in an exact way, we’re beginning to make progress (for example, we can make mice smell things and make humans see weird faces). 

The second method is unique to intelligent agents, and involves indirectly shaping them by controlling their experiences. This method works for intelligent agents (but not inanimate objects) because of the way experience necessarily shapes their internal structure (e.g. experiences driving new neural connections in the brain). We see this second method applied in the novel, with O’Brian shaping Winston through torture and conversation – no neuroscience required. Although controlling experiences doesn’t offer the same level of absolute control as directly rearranging the agent’s mind / brain, it’s a strategy which is significantly more achievable with our current understanding of the brain. To be successful, the shaper doesn’t need to (fully) know how the intelligent agent works at a deep level (e.g. the level of neurons), and can instead rely on heuristics to do the shaping (e.g. the idea that torture detaches people from their beliefs). The fact that this method is not exact is what makes O’Brian’s point interesting – to what degree can humans be shaped by their controlled experiences?

It seems the primary determinant of human malleability is age. As discussed here, tremendous changes happen in the brain during the first few years of life (allowing us to easily pick up language, social norms, etc.), with the rate of change slowing down considerably afterwards. In a sense, the brain “hardens” over time (or “anneals”, as discussed in the linked post), which makes it increasingly difficult to mold a particular way. It’s interesting to think about the degree to which someone could shape a person if they started from birth. We don’t have to look far to see incredible plasticity exhibited by early brains – the fact that any newborn can make any culture or language in the world their own, simply by growing up in it, highlights the flexibility of newborns. However, the fact that newborns can assimilate into any existing culture does not mean this holds true for any hypothetical culture, as it may be that the cultures which exist today exist because they’re particularly easy for humans to pick up. Terrence Deacon dives into this idea with regards to language in his book The Symbolic Species, examining the coevolution of humans and language and pointing out that just as our brains evolved for language, language evolved for our brains (and the same holds true for culture). Over time, those phonetic sounds and syntactical sequences which better “fit” our brains won out, resulting in languages built for our brains. 

This fact raises some interesting questions – for example, could a newborn learn a non-evolved language, such as Quenya (a language created by J.R.R. Tolkien)? I would imagine the answer is yes, firstly because Tolkien followed general language patterns in crafting the language, but also because the human brain seems to have a strong drive to pick up language (with specialized areas like Broca’s area and Wernicke’s area). I’m not sure how different a language would have to be for a newborn to be unable to pick it up if immersed in it – it’s difficult to imagine these alternate types of languages, due to the fact that all examples we have either evolved with humans or are human-crafted. The question of newborn flexibility is even more interesting when looking at culture. In 1984 we see the government working to create a culture of totalitarianism by indoctrinating children and instilling a primary allegiance to the state, and we’ve seen other governments attempt to do this throughout history. Historically, these attempts have seen limited success, usually due to the fact that they aren’t able to meet the basic needs of their citizens. While the higher-level thinking portion of someone’s mind might be manipulated in such a way as to “love Big Brother” (or love the government), it seems these feelings will give way to the more primal need for necessities such as food, water, and sex. Part of the “problem” is that people are generally intelligent enough to see that things could be different, and so they connect the dots and realize a different form of organization could serve them better. In 1984, the government takes steps to eliminate this “problem” by controlling and limiting education, ensuring that citizens only have enough knowledge and intelligence to play their required roles within society. However, even with this type of system, it seems there are still some parts of the mind which would be immutable.

As described above, certain feelings and desires for things like food and water seem to be too deeply rooted in our beings to be “molded out” of us. No matter what torture you subjected someone to, or what type of therapy you engaged with them on, it seems unlikely you could get the hunger impulse completely out of them (though you could likely change its strength). We could remove it using the type 1 methods described above (e.g. remove whatever neural circuits are responsible for sensing a lack of food and propagating the sense of hunger), but we can’t access it by subjecting certain experiences on someone. There’s a limit to how far experience can go in shaping a person – the brain is plastic, but only to an extent, and to a decreasing degree as we age. This is especially true when looking at feelings like hunger or thirst, but it extends to other realms, for example sex drive. In 1984, we see certain forms of prostitution or extramarital sex are tacitly allowed (though publicly condemned) as the government was unable to get them out of the populace. 

To us in the modern world, it feels like there are other drives which are impossible to “get out” of people – a desire for fairness, a primary concern with one’s own well being (and that of one’s family), a desire for progress and innovation, etc. However, it’s not clear whether these concepts just feel deeply rooted to us due to our upbringing, or if they’re a function of the innate structure of the human mind. In 1984, Winston feels these concepts, but Big Brother hammers them out of him – hopefully we never get to the point of running that experiment outside the book.