Excerpt

John(ny) von Neumann is most widely known for what’s called the von Neumann computing architecture. He never called it that, and in fact wrote about it only in an influential but fairly dreadful document (von Neumann 1945). That this was not a document for the ages is apparent in its title: “First Draft of a Report on the EDVAC.” It never got beyond a first draft, and it filters his thoughts on how to program a computer through the severe limitations of the system he was programming (1k memory, 1k operations per second).

This report enshrined a technological limitation that we’ve spent decades recovering from. The division of labor between expensive vacuum tubes and slow serial memories is mirrored in what can be thought of as a physics mistake in the Turing machine that influenced him (Turing 1937). This was a theoretical construct that introduced computational universality and undecidability, but in it the tape storing data is distinct from the head processing it. That’s unphysical: memory transistors are as universal as the transistors used in a processor, but data must be shuttled from the former to the latter to be operated in. Only recently have computer architectures overcome that bottleneck by mixing processing with memory.

Bibliography

Abdel-Rahman, A., C. Cameron, B. Jenett, M. Smith, and N. Gershenfeld. 2022. “Self-Replicating Hierarchical Modular Robotic Swarms.” Communications Engineering 1 (1): 35. https://doi.org/10.1038/s44172-022-00034-3.

Bennett, C. H. 1982. “The Thermodynamics of Computation—A Review.” International Journal of Theoretical Physics 21 (12): 905–940. https://doi.org/10.1007/BF02084158.

Deutsch, D., and C. Marletto. 2015. “Constructor Theory of Information.” Proceedings of the Royal Society A: Mathematical, Physical, and Engineering Sciences 471 (2174): 20140540. https://doi.org/10.1098/rspa.2014.0540.

Gershenfeld, N. 2020. “Scaling.” In Possible Minds: Twenty-Five Ways of Looking at AI, edited by J. Brockman. New York, NY: Penguin.

Gershenfeld, N., D. Dalrymple, K. Chen, A. Knaian, F. Green, E. D. Demaine, S. Greenwald, and P. Schmidt-Nielsen. 2010. “Reconfigurable Asynchronous Logic Automata: (RALA).” ACM Sigplan Notices 45 (1): 1–6. https://doi.org/10.1145/1707801.1706301.

Gershenfeld, N., A. Gershenfeld, and J. Cutcher-Gershenfeld. 2017. Designing Reality: How to Survive and Thrive in the Third Digital Revolution. London, UK: Hachette UK.

Griffith, S., D. Goldwater, and J. M. Jacobson. 2005. “Self-Replication from Random Parts.” Nature 437 (7059): 636. https://doi.org/10.1038/437636a.

Landauer, R. 1961. “Irreversibility and Heat Generation in the Computing Process.” IBM Journal of Research and Development 5 (3): 183–191. https://doi.org/10.1147/rd.53.0183.

Langton, C. G. 1984. “Self-Reproduction in Cellular Automata.” Physica D: Nonlinear Phenomena 10 (1–2): 135–144. https://doi.org/10.1016/0167-2789(84)90256-2.

—. 1995. Artificial Life: An Overview. Cambridge, MA: MIT Press.

Margolus, N. 1984. “Physics-Like Models of Computation.” Physica D: Nonlinear Phenomena 10 (1–2): 81–95. https://doi.org/10.1016/0167-2789(84)90252-5.

Penrose, L. S. 1959. “Self-Reproducing Machines.” Scientific American 200 (6): 105–117. https://doi.org/10.1038/scientificamerican0659-105.

Shannon, C. E. 1958. “Von Neumann’s Contributions to Automata Theory.” Bulletin of the American Mathematical Society 64 (3): 123–129. https://doi.org/10.1090/S0002-9904-1958-10214-1.

Turing, A. M. 1937. “On Computable Numbers, with an Application to the Entscheidungsproblem.” Proceedings of the London Mathematical Society s2-42:230–265. https://doi.org/10.1112/plms/s2-42.1.230.

—. 1952. “The Chemical Basis of Morphogenesis.” Philosophical Transactions of the Royal Society of London 237 (641): 37–72. https://doi.org/10.1098/rstb.1952.0012.

Ulam, S., H. W. Kuhn, A. W. Tucker, and C. E. Shannon. 1969. “John von Neumann, 1903–1957.” In The Intellectual Migration: Europe and America, 1930–1960, edited by D. Fleming and B. Bailyn, 235–269. Cambridge, MA: Harvard University Press.

von Neumann, J. 1945. First Draft of a Report on the EDVAC. Contract No. W-670-ORD-4926, June 30, 1945.

—. 1956. “Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components.” In Automata Studies, edited by C. E. Shannon and J. McCarthy, 34:43–98. Princeton, NJ: Princeton University Press. https://doi.org/10.1515/9781400882618-003.

—. 1966. Theory of Self-Reproducing Automata. Edited by A. W. Burks. Urbana-Champaign, IL: University of Illinois Press.

Zykov, V., E. Mytilinaios, B. Adams, and H. Lipson. 2005. “Self-Reproducing Machines.” Nature 435 (7039): 163–164. https://doi.org/10.1038/435163a.

BACK TO Foundational Papers in Complexity Science