The world of sport is filled with superstition. Michael Jordan famously wore University of North Carolina shorts under his Chicago Bulls kit; Serena Williams wears the same socks throughout a tournament; and when Goran Ivanisevic won a tennis match, he would repeat that day’s activities throughout the competition.
Psychologists say this behavior comes about because the human brain sometimes links events that have little or no causal connection. Computer scientists have a different way of thinking about it. For them, this is an example of “overfitting” — using irrelevant detail to construct a model. There may be many factors that contribute to the success of a particular tennis shot or basketball throw or home run but the color of socks or underpants is probably not one of them.
Exactly the same thing occurs with artificial neural networks. The networks learn relevant detail but also irrelevances. Indeed, overfitting is the bane of machine learning experts who have devised a wide range of techniques to get around it.
