Although GPT-3 was released ages ago (in AI time), it continues to generate interesting conversations, particularly with regard to the path toward general artificial intelligence. Building off a discussion of some others in the field (centered around the potential upside of scaling deep learning models), Scott Aaronson (a quantum computing expert who writes Shtetl-Optimized) and …
One important concept in data science is that of signal. Signal represents the ability of data to inform accurate predictions in the target domain; a dataset that can tell you more has more signal. For example, say you’re trying to predict the weather in New York for the next day (repeatedly). For the sake of …
I’m often torn between the competing ideas that: Understanding the brain, with its repeating structures and distinct modules, will be relatively easy (i.e. within the short-term grasp of humanity) Understanding the brain, with its structures crafted through millions of years of random evolution, will be relatively hard (i.e. will not happen for multiple generations) For …
A Circuit-Level View of Evolutionary Interpretability Read More »
We’ve now touched on the idea of general artificial intelligence (i.e. above human level) in a couple posts, and so it seemed time for a more thorough examination of how we might get there, and of what this type of “superintelligence” might look like. We’ll start out by taking a step back, and looking at …
A number of other posts so far have touched on what it is that brains do – and for the most part, it’s been summarized as “creating a model of the world”. By this, we’ve meant that certain patterns of neural activity can be understood as representing or standing for some observed pattern of material …
This post, and all posts published on this site, will be constructed using words. I imagine this seems reasonable enough (though there may be hope for some pictures as well!) – words are the foundation of communication across our world. But how well do we actually understand the nuance of this tool? In this post, …
It can be easy to take for granted all that our brains do when understanding the world, particularly in the realm of language. While many early philosophers recognized some of this complexity, the computer age brought about attempts to get computers to understand our communication tool – language – and these attempts brought home just …
Basketball hoops were invented by James Naismith Read More »