Since I started lifting weights, I’ve found curious the lack of any heuristics to calculate a one rep max based on a particular number of sets and reps. A number of calculations exist for a single set (see below chart from Wikipedia), but as pretty much all programming involves both sets and reps (e.g., bench …
I recently submitted an entry for the Inverse Scaling Prize, and while it wasn’t selected, I think it still reveals some interesting properties of model scaling that are worth exploring (and are similar to those analyzed in one winning submission from Cavendish Labs). The goal of the competition is to identify machine learning tasks where …
Although GPT-3 was released ages ago (in AI time), it continues to generate interesting conversations, particularly with regard to the path toward general artificial intelligence. Building off a discussion of some others in the field (centered around the potential upside of scaling deep learning models), Scott Aaronson (a quantum computing expert who writes Shtetl-Optimized) and …
“Learning” is another one of those abstract concepts which reveals significant complexity upon further examination. In the context of people, learning represents our ability to incorporate experience in a beneficial way; we can learn facts, skills, or social norms (among countless other things) through repeated (or one-time) exposure. The exact mechanics underlying the learning process …
With the massive degree of progress in AI over the last decade or so, it’s natural to wonder about its future – particularly the timeline to achieving human (and superhuman) levels of general intelligence. Ajeya Cotra, a senior researcher at Open Philanthropy, recently (in 2020) put together a comprehensive report seeking to answer this question …
Examining Evolution as an Upper Bound for AGI Timelines Read More »
As previously discussed in this post, deep learning models designed to predict and create natural language have become quite powerful, and are now able to do things like write news articles or compose poetry at a level similar to (or even better than) the average human. This is a significant achievement which will likely have …
Deep Learning Language Models and Exact Procedures Read More »
The field of machine vision has progressed rapidly over the last decade, with many systems now achieving “better than human” results on standardized image recognition tests. Deep convolutional neural networks have been a main driver of these improvements, and have been enabled by increasing data availability and computing power. ImageNet Competition Best Error Rate Performance, …
A new natural language AI model launched by OpenAI, GPT-3, has been making waves in the artificial intelligence community. GPT-3 is a transformer model (simplifying greatly, a neural network approach with modifications for better performance on text) trained on one specific task: predicting the next word, given all previous words within some text. This simple …
A number of other posts so far have touched on what it is that brains do – and for the most part, it’s been summarized as “creating a model of the world”. By this, we’ve meant that certain patterns of neural activity can be understood as representing or standing for some observed pattern of material …