Yes, they did. MuZero was published just before Christmas 2020, and is arguably the most advanced artificial neural network based on reinforcement learning out there right now. Its predecessor, AlphaZero already had the ability to teach itself both chess and Go (another two-person zero-sum-game without random elements). AlphaZero beat world champions, but needed one important bit of information to learn: the rules of the game. Not so MuZero. MuZero learned to play without data input, with knowledge of previous games, and without knowledge of the rules of the game. How is this possible?
MuZero is the latest in a series of developments that started with the 1958 Perceptron, the first computer-simulated neural network. For many decades, neural networks were sidelined in AI research. The breakthrough only came surprisingly recently, less than 10 years ago. In 2021, artificial neural networks are all but synonymous with AI. Deep Learning refers to the key improvement of the 1958 Perceptron: many neuron layers (where the Perceptron had only one). But the AIs that predict what friends you really want to have and what soap you really want to buy are all based on massive amounts of input data. By contrast, MuZero truly learns all by itself. It does so based on reinforcement learning, a method of self-learning that is reward-driven and goal-driven. For MuZero, reinforcement learning has been upgraded with the ability to form predictive models of what it is trying to learn. Isn’t this yet one step closer to the way we learn?
Yes, probably. But there remains a fundamental difference, the subject of The Self-Assembling Brain: biological brains grow based on genetic information. MuZero, like its predecessors, is a designed network with initially random connectivity that is switched on to learn. Babies are not born with randomly connected brains and switched on to learn. MuZero was of course not designed to simulate babies, but to do one thing: play games like chess. There are many more applications for a self-learning net like MuZero, and progress will surely be fast and astonishing. Yet, the paper avoids to discuss so-called ‘human-level intelligence’. For good reasons. The Self-Assembling Brain puts forward one reason that has not entered the mainstream of AI research: biological brains are smart long before they learn anything, they grow smart. The implications of development are worth a discussion.