This is a follow-up on my earlier post titled “No useful theory of biological neural computation yet“.
A recent Quanta article titled “Artificial Neural Nets Finally Yield Clues to How Brains Learn” by Anil Ananthaswamy gives me hope that Neuroscientists are getting closer to having a rudimentary model of biological neural computation. We are still far away from such a theory but there is progress.
Central question (posed in the Quanta article): Is it possible for the brain to use a mechanism similar to the backpropagation algorithm used in machine learning?
Answer (message of the Quanta article): There may be different ways the brain could be doing backpropagation. One of those ways may involve pyramidal neurons.
What is backpropagation?
“The algorithm enables deep nets to learn from data, endowing them with the ability to classify images, recognize speech, translate languages, make sense of road conditions for self-driving cars, and accomplish a host of other tasks.” – Quanta article
According to the abstract of the original paper on backpropagation [1] from 1986:
“We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure.”
The tutorial on deep neural networks by Carlos F. Crispim-Junior explains the subject with many visuals.
Why the backpropagation algorithm represented a revolution in machine learning?
Short answer is efficiency. The backpropagation algorithm is more efficient in the classification task compared to other algorithms. Note, however, that the brain is several orders of magnitude more efficient than any known machine learning algorithm. That’s why many scientists doubt that the brain is using backpropagation. Scientists are investigating biologically plausible algorithms such as feedback alignment, equilibrium propagation and predictive coding.
References
[1] D. Rumelhart, G. Hinton, & R. Williams, “Learning representations by back-propagating errors“. Nature 323, 533–536 (1986). https://doi.org/10.1038/323533a0
Suresh Emre
Renaissance Universal 