Thinking Bottom-Up

Humans (and all other brain-having organisms) interact with the world in a hierarchical manner, starting from the top and working down. When we decide to engage in an action, we make the decision at the highest level (for example, deciding to write a blog post), and then break up that action into its lower level parts (writing the first paragraph, which requires writing the first letter, which requires moving a hand to hit the key, etc.). Our species has become adept at this manner of solution, using it to engineer a wide range of complex artifacts, from skyscrapers to spaceships to computers. However, not all problems yield equally well to this approach. Effective top-down design requires independent (or mostly independent) sub-problems, and rapidly becomes ineffective as dependencies are introduced. These dependencies can impede progress even on simple problems, like solving a Rubik’s cube. The overall problem seems straightforward (make each face one color), as do the sub-problems (make a particular face one color), but progress towards one sub-goal interferes with the state of others, commonly leading to great difficulties solving it (unless you know the algorithms). While the top-down approach runs into issues, there’s another way to solve this class of problems – the bottom-up approach. Bottom-up design involves starting from the lowest relevant level (in the case of the Rubik’s cube, the individual stickers) and understanding the interactions between the parts, and how those interactions drive higher level behaviors. While it’s not our forte, we can identify solutions to problems like the Rubik’s cube with the right mathematical approach (hence the algorithms to solve it). However, as further complexity is added, problems can quickly outstrip the capabilities of our top-down oriented minds.

While we humans struggle with bottom-up engineering, it’s Nature’s bread and butter. The natural world is fully parallel, with all sorts of particles bumping into other particles all at the same time, coalescing into groups of particles interacting with other groups, and those into groups of groups, ad infinitum. This hierarchy of patterns is Nature’s canvas, and on it she has painted a great range of complex phenomena, from atoms and molecules to DNA and humans. “Luckily” for us (and other macroscopic life), much of what she has painted at our level of experience is regular and understandable; trees, lakes, mountains, and even other animals all lend themselves to comprehension. Our ancestors (or really, their brains) latched onto these regularities to better engage with the world, developing a top-down approach to do so. This regularity-comprehension was a “good trick” which aided survival and replication, and so the capabilities continued to expand, eventually leading to the evolution of homo sapiens. This species brought with it a paradigm shift, as their brains’ powers of representation were such that they started to represent concepts of far greater complexity than the trees and the lakes; they formed conceptions of themselves, their history, and their means of functioning. They (we) can see enough of Nature’s painting to recognize the layers, and exploration of these deeper layers has been a central activity of our species (first through myth, and now with science). However, Nature’s bottom-up methods frequently impede our exploration and understanding. This hindrance was especially evident in our attempts to understand the diversity of life and how we came to be, and is evident now in our attempts to understand the brain. The rest of this post will examine the leap Darwin made to answer the former question, and how it might help us think about the latter.

The question of life’s diversity on earth and the origin of humans had been asked for thousands of years before Darwin came up with the answer. For much of that time, the accepted answer was that we were designed (though the specific designer varied by culture). Design – more specifically, “intelligent” design – was all we knew, as it was how we engineered things ourselves. The bottom-up explanation eluded us even as we built up a significant fact base on the species of the world (including extinct species with fossil records) and made progress in grouping certain types together. The non-mythological hypotheses which did gain traction were reflective of our top-down manner of thinking, such as the idea that an innate life force drives organisms to become more complex over time (Lamarck’s transmutation of species). In thinking about our origins, our imaginations only waded through the realm of top-down design.

Darwin’s great insight was to flip the design process on its head, and recognize the powers and possibilities of a bottom-up approach. He saw that Nature didn’t need to “know” what it was painting; all it needed was the right sets of conditions to constrain the process. The list of conditions required ended up being fairly simple: variation, heredity, and competition. In simple terms, variation means organisms have different characteristics, heredity means these different characteristics are passed down to children, and competition means resource constraints will limit survival and reproduction. Darwin realized that, taken together, these conditions meant that the organisms “better fit” for their environment would be the ones to survive and procreate. Nature can drive the process forward simply by passively selecting (“Natural Selection”). The significance of Darwin’s inversion of logic cannot be understated. Prior to Darwin, our species lacked any conception of bottom-up design; we only knew that humans could engineer artifacts, and thus the working assumption was that all artifacts (in the broad sense) must be engineered by intelligence (again, in the broad sense – Lamarck’s innate life force can be viewed as “intelligent” in that it “knew” which way to progress the organism). We saw the vastness of the space of possible designs, and thought that vastness meant intelligence was required for the creation of complex artifacts. Darwin realized that, while the design space was certainly vast, a particular set of simple conditions would add a direction to progress. Intelligence was not required to produce intelligence.

Taking a slightly more mathematical perspective, Darwin’s conditions can also be thought of as imposing a gradient on the exploration of the design space, constraining the process to move in the direction of “better fit” organisms. To fully understand this effect, it can be helpful to take a step back and try to visualize the design space. The total space is exceedingly vast – there’s an essential infinity of possible arrangements of matter within our universe (stretching from a single atom of hydrogen to an earth-size replica of the periodic table made of the respective elements, and beyond). Within this vast space, however, only an infinitesimally small portion of arrangements are stable over time, with an even smaller portion capable of replicating themselves. We know this portion as life. Of these possibilities of life, the forms present on earth (based on the DNA system of replication) are again an infinitesimally small portion of the total possibilities.

Darwin’s conditions act as an arrow directing the progress of life; specifically, they guide the progression toward “better fit” states. Here on earth, the design space has been limited to that of DNA-based forms (though we can imagine others), but even within this more limited space there are an essentially infinite number of possibilities. Humans, for instance, are coded with about three billion nucleotides – the four options for each (“A”, “C”, “G”, or “T”) mean there are ~10^1,800,000,000 possibilities! While most of these would not produce life (i.e. a persisting and replicating organism), many would, and we can imagine each having a “fitness” score, based on the environment it occupies. Crucially, no “scorer” is required to assign fitness scores; instead, organisms are “scored” over the course of their lives by the natural goings of the world. We can plot this design space / fitness score relationship – for simplicity, we’ll use a 2D representation, although in reality the design space is many-dimensional.

This simplified chart highlights the relationship between the design space and the fitness score – different designs have different levels of fitness. Going a step further, we can add a population of organisms to the chart.

We can see the organisms vary in relative fitness due to slight differences in design. This is not a deeply insightful observation, and it would have been well understood pre-Darwin. The critical question was how populations moved along the curve – essentially, how did organisms get “better”? Pre-Darwin theories looked at the fitness evaluation from the top-down; someone or something had to “know” what was a better fit, whether that was a god or some innate life force (Lamarck). Darwin saw that no one needed to know. Instead, the right set of conditions formed a gradient, driving movement along the design space in the direction of increased fitness.

It’s interesting to note that, with the innumerable dimensions and possibilities of the biological design space, a bottom-up approach is the only realistic path. A top-down approach requires, at minimum, directional understanding of the relationship between designs and their fitness. When we build a computer, we understand the role each part will play, and can combine parts using that knowledge. When considering life, on the other hand, there’s no clear way to understand the fitness of a sequence of DNA without actually “running the program”. We may be able to understand the proteins coded for by a particular sequence, but understanding fitness would require knowledge of how all the parts would come together to form the organism (and additionally would require an understanding of how that organism would fare in its environment). The bottom-up approach, rather than try to parse out a relationship, simply imposes a gradient towards greater fitness. This idea may seem obvious in retrospect, as it fits together so simply (though the exact details of implementation are complex and still a target of research). It took so long to come up with because our brains don’t think in a bottom-up manner; for all their parallelism, their thinking can be frustratingly sequential. 

Our species is currently engaged in an endeavor strikingly similar to Darwin’s – understanding the brain. We have a great deal of data on neurotransmitters and neurons and circuits and regions, but no solid theories of how it all comes together to drive intelligence. We seem to be in need of another Darwin. Lacking that, let’s take a look at how we might think about the workings of the brain from his perspective.

To start, let’s review how our brains form. We are born with ~100 billion neurons and ~2,500 synapses (connections) per neuron, but the number of synapses per neuron jumps to ~15,000 by age two or three. From there, neuron and synapse counts both decrease, with ~86 billion neurons and ~10,000 synapses per neuron in the adult brain. To summarize, we’re born with a large number of neurons but few synapses, then grow synapses rapidly until age two or three, from which point neurons and synapses are both culled. We also know that our understanding of the world continually improves during this development; we use our senses to learn a great deal about its regularities (starting with classes of objects, and eventually progressing to abstract concepts and one-shot learning). How does the development process of billions of individual neurons drive this understanding of the world? 

 We can imagine the design space of the neurons – equally vast and difficult to parse as that of the genome (consider that we have ~100 billion neurons, each connected to up to 10,000 others). Within that vast design space, there exist configurations that understand and are aligned with the world (in fact, there exist configurations that understand quantum physics, others that know 10,000 digits of pi, and others that feel compelled to write posts on bottom-up design, though most are non-functional). Finding a good configuration requires a similar approach as for the evolution of organisms; the characteristics of the neurons must be such that over time they move toward better configurations, where here “better” means more accurately making sense of the incoming inputs (seeing, hearing, etc.) Simply put, the gradient must point in that direction.

 The job of the neuroscience theorist is to come up with the conditions which lead that to be the case (the brain equivalents of variety, heredity, and competition). We currently have a limited understanding of a couple: we know that “neurons that fire together, wire together” (Hebb’s rule), and that neurons and synapses which are rarely used generally die (especially during the culling after age three). However, greater specificity and additional conditions are required for us to understand the processes of the brain in an implementable way. What else do we need to know? I’ve listed some thoughts below – please feel free to extend the list in the comments section!

• How do neurons need to be arranged (e.g. cortical columns) for the process to be effective?
• How do these arrangements work together? 
• What role do different types of neurons play?
• What is the role of neural firing vs. firing frequency?
• How do new concepts “know” where to be represented? (this is more a top-down question)
• What types of global signalling (e.g. dopamine release) are involved and how do they influence neuron behavior?
• What types of competition occur between neurons?

There’s no reason to think the conditions required for the brain to move towards a “better” fitness state will be as simple as those required for evolution. However, we do have reason to believe describable conditions exist, as the flexibility of our intelligence combined with the limited informational capacity of our genes requires a general approach. Let’s hope these conditions are powerful enough to let our brains figure them out.

Author’s Note: If you enjoyed this post, please consider subscribing to be notified of new posts 🙂

Name*

Email*