In the prologue to his 2020 book, The Alignment Problem: Machine Learning and Human Values, Brian Christian tells the story of the beginnings of the idea of artificial neural networks. In 1942, Walter Pitts, a teenage mathematician and logician, and Warren McCulloch, a mid-career neurologist, teamed up to unravel the mysteries of how the brain worked. It was already known that neurons fire or do not fire due to an activation threshold.
"If the sum of the inputs to a neuron exceeded this activation threshold, then the neuron would fire; otherwise, it would not fire," explains Christian.
McCulloch and Pitts immediately saw the logic in the activation threshold — that the pulse of the neuron, with its on and off states, was a kind of logic gate. In the 1943 paper that came out of their early collaboration, they wrote, "Because of the 'all-or-none' character of nervous activity, neural events and the relations among them can be treated by means of propositional logic." The brain, they realized, was a kind of cellular machine, says Christian, "with the pulse or its absence signifying on or off, yes or no, true or false. This was really the birthplace of neural networks."
