A Circuit-Level View of Evolutionary Interpretability

I’m often torn between the competing ideas that:

  1. Understanding the brain, with its repeating structures and distinct modules, will be relatively easy (i.e. within the short-term grasp of humanity)
  2. Understanding the brain, with its structures crafted through millions of years of random evolution, will be relatively hard (i.e. will not happen for multiple generations)

For the sake of productivity, I generally find myself leaning toward option 1 (and would imagine neuroscientists lean that way all the harder), but sometimes it’s tough to resist the pull of option 2, especially when a compelling analogy comes along. I had read a shorter version of this story a while ago, but came across it again recently and felt it strongly connected to the option 2 argument. To summarize the story (though I recommend reading the linked article), Dr. Adrian Thompson was a researcher who wanted to test out whether ideas from evolution could help with the design of circuits. He set up an experiment in which a large number of circuits were evaluated in their ability to distinguish two sound tones, with the best ones then swapping parts of their configurations (similar to the mixing of genes that happens with sexual reproduction), together with the occasional introduction of random mutations. Importantly, he ran this experiment using extremely small circuit sizes (relative to the problem) – a normal circuit for sound differentiation might have hundreds of thousands or millions of logic gates (i.e. AND, NOT, etc.), while Thompson allowed his circuits only 100 (and no clock as would be typically used for synchronization). The outcome of this experiment was extremely interesting, as the article linked above describes:

Dr. Thompson peered inside his perfect offspring to gain insight into its methods, but what he found inside was baffling. The plucky chip was utilizing only thirty-seven of its one hundred logic gates, and most of them were arranged in a curious collection of feedback loops. Five individual logic cells were functionally disconnected from the rest⁠— with no pathways that would allow them to influence the output⁠— yet when the researcher disabled any one of them the chip lost its ability to discriminate the tones. Furthermore, the final program did not work reliably when it was loaded onto other FPGAs of the same type.

It seems that evolution had not merely selected the best code for the task, it had also advocated those programs which took advantage of the electromagnetic quirks of that specific microchip environment. The five separate logic cells were clearly crucial to the chip’s operation, but they were interacting with the main circuitry through some unorthodox method⁠— most likely via the subtle magnetic fields that are created when electrons flow through circuitry, an effect known as magnetic flux. There was also evidence that the circuit was not relying solely on the transistors’ absolute ON and OFF positions like a typical chip; it was capitalizing upon analogue shades of gray along with the digital black and white.

Even this simple process, with the configuration of only 100 logic gates evolving over several thousand generations, led to a nearly uninterpretable result. Thompson was able to identify at a high level the way the circuit handled the task (through experimentation with removing the separate logic cells), but performing a deeper analysis of how the circuit actually recognized the tones would have required diving into some extremely complex analysis (due to the analog nature of the design). Evolution has no requirement of making its products interpretable.

When looking at our brains, we have a far more complex system (with billions of neurons) which evolved in a far more complex environment (the natural world), over a significantly greater number of generations, which does not bode well for interpretability. The less complex brains of more primitive organisms offer an even more striking comparison to Thompson’s circuit; C. elegans, a tiny roundworm, has only 302 cells in its nervous system (each worm has the same number and same network of connections), and yet precise understanding of how these neurons come together to drive behavior has continued to elude researchers. 

It seems our brains work top-down, looking for broad patterns in the world, and then for sub-patterns when required (and so on, down the hierarchy). To build a circuit, we first lay out the overall architecture, then work on the specifics of each module in the architecture (separately), then implement those structures using gates. This stands in stark contrast to the evolutionary strategy, which starts from the lowest level and simply iterates until high-level success is achieved. It is interesting that the mechanisms which enable our top-down intelligence have been constructed bottom-up, with the designs forged through evolutionary selection (even the development of a single brain can be viewed through the evolutionary lens, with far more neurons and synapses forming than required, leading to a “trimming down” based on which circuits are “active”). Unfortunately, this difference in approach may also mean a long journey toward understanding.

5 1 vote
Article Rating
Subscribe
Notify of
2 Comments
Inline Feedbacks
View all comments
Jon
2 years ago

I read the story attached, very interesting experiment with an unexpected result. I like what you said “Evolution has no requirement of making its products interpretable.” Seems like there’s too much going on in our brains for no real reason other than thats how it was molded from the bottom up, that we can never fully understand it. Kind of a cool thought how the brain can’t understand itself, but it makes us conscious brings capable of extreme intelligence and able to observe and think about the world around us