Foundational Papers in Complexity Science pp. 89–113
DOI: 10.37911/9781947864528.05
The Genesis of Modern Computing and AI
Authors: Bruno Olshausen, University of California, Berkeley; Redwood Center for Theoretical Neuroscience; and Santa Fe Institute; and Christopher Hillar, Redwood Center for Theoretical Neuroscience
Excerpt
Digital computers and neural networks would seem to be profoundly different systems for computation: one represents information with strings of 1s and 0s, the other as a set of analog values. One manipulates these representations through a cascade of logical operations coordinated by a central clock and processing unit, while the other transforms representations via matrix-vector multiplications and thresholding operations that run in loosely coordinated, parallel streams. The first served as the workhorse of the computing industry for more than half a century, while the second has only recently seen widespread deployment as the new computing paradigm powering artificial intelligence. Given such stark differences—both in their computational architectures and the technologies they have enabled—it may come as a surprise to learn that both architectures share a common origin in this landmark paper.
Published in 1943 amidst a world in upheaval, Warren McCulloch and Walter Pitts’s paper presented a highly original and thought-provoking hypothesis: that human mental abilities, especially our capacity for logical thought, stem directly from neuronal circuits in the brain that themselves perform logical operations. While others such as Turing had previously speculated about mental processes as a kind of computation, no one had attempted to develop a computational theory of mind rooted in the biophysical substrates of the brain—that is, neurons. Their paper thus marks an important turning point in our intellectual approach to studying and understanding the brain. It is the product of an interdisciplinary collaboration born from a coming together of minds at the University of Chicago, where a revolutionary new movement in “mathematical biology” was taking place under the guidance of Nicolas Rashevsky. The premise of their approach was that mathematical models could be used to gain new insight into biological processes in much the same way that they had enabled powerful theoretical advances in physics (Abraham 2002).
Bibliography
Abraham, T. H. 2002. “(Physio)Logical Circuits: The Intellectual Origins of the McCulloch–Pitts Neural Network.” Journal of the History of the Behavioral Sciences 38 (1): 3–25. https://doi.org/10.1002/jhbs.1094.
Arathorn, D. W. 2002. Map-Seeking Circuits in Visual Cognition: A Computational Mechanism for Biological and Machine Vision. Redwood City, CA: Stanford University Press.
Gefter, A. 2015. “The Man Who Tried to Redeem the World with Logic.” January 29, 2015, Nautilus, https://nautil.us/the-man-who-tried-to-redeem-the-world-with-logic-235253/.
Godfrey, M. D. 1993. “Introduction to “First Draft of a Report on the EDVAC” by John von Neumann.” IEEE Annals of the History of Computing 15 (4): 27–75. https://doi.org/10.1109/85.238389.
Hinton, G. F. 1981. “A Parallel Computation that Assigns Canonical Object-Based Frames of Reference.” In IJCAI ’81: Proceedings of the 7th International Joint Conference on Artificial Intelligence, Vancouver, BC, Canada, 683–685. San Francisco, CA: Morgan Kaufmann Publishers, Inc.
McCullough, D. 2015. The Wright Brothers. New York, NY: Simon & Schuster.
Olshausen, B. A., C. H. Anderson, and D. C. Van Essen. 1993. “A Neurobiological Model of Visual Attention and Invariant Pattern Recognition Based on Dynamic Routing of Information.” Journal of Neuroscience 13 (11): 4700–4719. https://doi.org/10.1523/JNEUROSCI.13-11-04700.1993.
Piccinini, G. 2004. “The First Computational Theory of Mind and Brain: A Close Look at McCulloch and Pitts’s ‘Logical Calculus of Ideas Immanent in Nervous Activity.’” Synthese 141:175–215. https://doi.org/10.1023/B:SYNT.0000043018.52445.3e.
Pitts, W., and W. S. McCulloch. 1947. “How We Know Universals; The Perception of Auditory and Visual Forms.” The Bulletin of Mathematical Biophysics 9:127–147. https://doi.org/https://doi.org/10.1007/BF02478291.
Rosenblatt, F. 1958. “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain.” Psychological Review 65 (6): 386–408. https://doi.org/10.1037/h0042519.
—. 1962. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Vol. 55. Washington, DC: Spartan Books.
von Neumann, J. 1945. First Draft of a Report on the EDVAC. Technical report. Contract no. W-670-ORD-4926 between the United States Army Ordnance Department and the University of Pennsylvania. Moore School of Electrical Engineering, University of Pennsylvania.