Imagine we have a man in a room, and we can ask him to perform any task (we’ll assume he’s motivated to do the tasks). These tasks range across the entire domain of the world, for example:
- Shoot a bow and arrow at a target (given a bow and arrow and a target)
- Point to the second tallest tree a picture (given a picture of trees)
- Identify the type of tree in the picture (given a book on tree types)
- Jump rope (given a rope)
- Answer which is heavier, a mouse or a cat
- Sing “Happy Birthday”
- Write an algebraic equation which has “0” as a solution (given a book on algebra, if necessary)
- Cook a meal (given ingredients and a kitchen)
- Drive a car (given a car and a road)
- Serve as a cashier (given training and a register)
- Etc. (infinite possibilities)
We can imagine that the man would perform fairly well on these tasks, and given sufficient time could succeed on all of them. Even on classes of tasks the man has not seen or heard of before, he can perform well, as he has a sufficiently powerful world model to make sense of the words and the task. This man is a standard example of general intelligence; he can accomplish an essentially infinite number of types of goals, as he starts with a world model and then incorporates the goal into that world model.
How might we go about creating a computer program to replace the man? On any one of these tasks, we could figure out a way to program a computer to do it; in fact, we even have systems which could accomplish whole classes of tasks (for example, GPT-3 could likely cover most non-exact language questions). The issue for computers here is that there is no single task-level goal (or even class of goals) – this system will only have a global “goal”, which is to be good at accomplishing task-level goals. This mandate doesn’t give us anything to “grab onto” algorithmically when building a system; there are no shortcuts. We can’t use our traditional machine learning methods, as there is no specific task-level goal to target (and the global goal domain spans the entire natural world). It seems the only viable path is to construct a system that effectively generates a world model (together with the behavioral drivers necessary for cooperation) and then accomplishes the specific tasks by using its global model to “understand” the tasks and convert them into behavior or output (similar to the man). There seems to be no room for specifics at the base of this generally intelligent system, no way of “inserting” individual goals; all we have at the base is the kernel of world modeling. The only way to get these goals “into the system” is from the top down, once the system has a sufficiently powerful world model to “understand” the requested task (and even then, the system will “understand” the goal in its own way, with its own concepts, based on the world model it has developed).
Does this resonate with how others think about AGI vs. AI? In reading literature on the topic, many seem to treat AGI as “scaled up” AI – for example, Bostrom writes extensively about an AGI built around the final goal of maximizing paperclips in his book Superintelligence. As I see it, you could ask the system to make paperclips, or build pressures into the system (similar to how humans are pressured to have sex), but if the system is truly a generally intelligent one you could not encode the goal at the lowest level (unless you created a kernel to world modeling which also drove paperclip-making – but this seems nearly infinitely harder than creating a kernel geared only to world modeling). Very interested to hear how others see it – please feel free to continue the conversation in the comments section below!
Author’s Note: If you enjoyed this post, please consider subscribing to be notified of new posts 🙂
couldn’t something like gpt-3 be used to generate a effective world model
Potentially, but only in a significantly different form than it exists today. GPT-3 works by predicting the next word, given previous words of text. Due to the nature of this task, there’s an extremely large amount of data out there to train from, and so GPT-3 finds a very good error minimizing location within the domain. Extending that concept to the domain of the natural world, we’d want a GPT system which took in “senses” and predicted what it would “sense” next. For this system to do things, we’d also want to give it some effectors (wheels, arms, etc.). It’s… Read more »
It seems, the more general the capabilities of the AI, the more specific the goal will need to be spelled-out. Does that seem realistic? I say that bc with generality comes potential…the limitations are reduced a lot, in order to allow for the AI to do so many more tasks. Is this in-line with your, “There seems to be no room for specifics at the base of this generally intelligent system, “? i.e. Are there different considerations for AGI vs ANI, when we grant access to a bow and arrows? With ANI we’ve imposed limits on what it should not do… Read more »
I think I’m on the same page as you. It seems as we move toward more generally intelligent systems, the manner in which we specify the goal changes. For a chess playing system, the goal is part of the creation of the system; we build the system specifically around the goal, and once it is built there is no question of “getting it to play chess”, for all it can do is play chess. For a more complex system like GPT-3, it’s not as straightforward to “give it a goal” – it can actually take a great deal of effort… Read more »
I can understand that…and the more i learn, the more i want to ban the use of the word “maximizer” from any discussio of real-world AI, lol…seems incredibly detrimental, the more we talk about generalization…with an ANI, then maximal effort is usually safely constrained by the environ we’ve predetermined…but how can anything intelligent come from trying to specify a goal with “maximizes” anything–it seems to, by definition, work directly against balancing other considerations inherent in the goal (which I also maintain –so far–cannot be a singular one)
I agree! Very important to define terms precisely (and perhaps avoid “maximizer”, as prior notions can result in confusion).