An Artificial Neural Network Forms Its Own Memories

An artist's conception of a differentiable neural computer. The neural network at the center does the data parsing, while reading writing and rewriting its memories. (Credit: DeepMind) Clive Wearing is a noted British musician, but he’s perhaps best known as the man with a 30-second memory. In the 1980s, Wearing contracted a strain of herpes virus that attacked his brain and destroyed his ability to form new memories. He might forget what he’s eating before food reaches his mouth. He struggles to frame experiences of the present with conceptions of time and place. Life for him is often akin to waking up from a coma — every 20 seconds. In a certain sense, artificial neural networks are Clive; they operate without working memory, erasing everything they learned when assigned to a new task. This limits the complexity of operations they can accomplish, because in the real world, countless variables are in constant flux. Now, the team from Google DeepMind has built a hybrid computing system, what they’re calling a “differentiable neural computer” (DNC), which pairs a neural network with an external memory system. The hybrid system learned how to form memories and use them to answer questions about maps of the London Underground transit system and family trees. “Like a conventional computer, it can use its memory to represent and manipulate complex data structures but, like a neural network, it can learn to do so from data,” the authors wrote in their paper, which was published Wednesday in the journal Nature.