When digital computers appeared in the 1950s, two main approaches emerged about how they should be understood. This debate led to deep and important discussions. One group thought that computers were physical symbol systems that could instantiate formal representations of the world. The other group saw them as simulations of neuronal interactions. One approach was framed in terms of problem solving, the other in terms of learning. One emphasized logic, the other statistics (8).
But there was also a second intuition, based in particular on the ideas of Hebb and McCulloch. “It was directly inspired by the work of D.O. Hebb, who in 1949 suggested that a mass of neurons could learn if when neuron A and neuron B were simultaneously excited, that excitation increased the strength of the connection between them” (7, 18). This led Rosenblatt to the idea that, if intelligent behavior was difficult to formalize, AI could instead be investigated by studying how neurons learn to recognize patterns and suggest responses.
At first, both approaches seemed equally powerful. But by the 1970s, the connectionist approach, which took the perceptron as its foundation, lost popularity. One reason was that the computers of that time could handle only simple tasks and failed at more complicated ones. This decline is sometimes referred to as the “AI winter.” Today’s multilayer perceptrons, however, can solve difficult problems and have contributed to the current rise of AI. Rosenblatt himself, however, spoke mainly about the single-layer perceptron.
Mark I Perceptron
The analogy between the simple on/off units of digital computers and the neurons of biological systems inspired theorists from many areas. In 1957, Frank Rosenblatt wrote a research proposal called Project PARA (Perceiving and Recognizing Automaton). This led to the creation of the first artificial neural network—the Mark I Perceptron (5).
This work was based on the simplified mathematical model of a neuron proposed by Warren McCulloch and Walter Pitts (4), which became a key inspiration for early neural network research. A second source was Canadian psychologist Donald Hebb’s The Organization of Behavior (3), which proposed a theory to explain how neurons could account for both perceptual generalization and the stability of memory.
Rosenblatt’s idea was to understand the problem of perception and memorization in biological networks: how information is sensed, stored in memory, and influences recognition and behavior. He stated that the analogy between biological and artificial systems should be apparent: the perceptron was “designed to illustrate some of the fundamental properties of intelligent systems in general, without becoming too deeply enmeshed in the special, and frequently unknown, conditions which hold for particular biological organisms” (6, 387).
The system was described as composed of three layers: the sensory or S-System, an association or A-System, and finally the response or R-System. The S-System could be imagined as a set of points on a TV raster or a collection of photocells; the R-System as output devices like type-bars or signal lights; and the A-System as the core that transmitted and transformed input signals using adjustable thresholds (2).
The Convergence Theorem
Rosenblatt also formulated his famous Convergence Theorem:
“Given an elementary oc-perceptron, a stimulus world W, and any classification C(W) for which a solution exists; let all stimuli in W occur in any sequence, provided that each stimulus must reoccur in finite time; then beginning from an arbitrary initial state, an error correction procedure (quantized or non-quantized) will always yield a solution to C(W) in finite time, with all signals to the R-unit having magnitudes at least equal to an arbitrary quantity <f * 0” (7, 11).
In simpler terms: if a problem can be solved by a perceptron, then the perceptron will always find a solution after enough training examples, using its error-correction rule. This meant that perceptrons could reliably learn classifications—within their limits.
The weights that the perceptron learned from samples of training data were literally “weights,” since they represented weighted connections between mechanical devices. The A-System provided the perceptron’s memory, but what exactly it was “remembering” in these weights was open to debate (1, 42).
Rosenblatt was remarkably optimistic. He wrote that even a perceptron “with a single logical level of A-units and response units” could already support pattern recognition, associative learning, selective attention, and recall. He also suggested that perceptrons might extend to temporal as well as spatial pattern recognition, and across multiple sensory modalities (6).
Unlike digital computers, which stored exact representations, the perceptron did not simply memorize inputs. Instead, its weights stored generalized information that allowed it to separate categories. For Rosenblatt, this was a sign of success: the perceptron demonstrated Hebb’s theory in practice by distinguishing between classes of data without memorizing the individual inputs (1, 43).
He even asked:
“Does this mean that the perceptron is capable, without further modification in principle, of such higher order functions as are involved in human speech, communication, and thinking?” (6, 404).
Minsky’s Critique and the AI Winter
However, Marvin Minsky and Seymour Papert offered a skeptical response. Advocates of the symbolic approach, they argued that perceptrons, considered as pattern-recognition and learning machines, had little scientific value. Their famous book showed that single-layer perceptrons could not solve certain problems (like XOR). But this critique ignored Rosenblatt’s own writings on multilayer perceptrons, presented in Principles of Neurodynamics (7).
This debate—connectionism versus symbolism—was probably the first of its kind in AI history. It ultimately contributed to the first “AI winter.”
Bibliography
- Dobson, James E. “Memorization and Memory Devices in Early Machine Learning.” Interfaces: Essays and Reviews on Computing and Culture, vol. 4. Minneapolis, MN: Charles Babbage Institute, University of Minnesota, 2023, pp. 40–49.
- Dobson, James E. The Birth of Computer Vision. Minneapolis, MN: University of Minnesota Press, 2023.
- Hebb, Donald O. The Organization of Behavior: A Neuropsychological Theory. New York: John Wiley & Sons, 1949.
- McCulloch, Warren S., and Walter Pitts. “A Logical Calculus of the Ideas Immanent in Nervous Activity.” The Bulletin of Mathematical Biophysics, vol. 5, 1943, pp. 115–133. https://doi.org/10.1007/BF02478259
- Rosenblatt, Frank. The Perceptron: A Perceiving and Recognizing Automaton (Project PARA). Report No. 85-460-1. Buffalo, NY: Cornell Aeronautical Laboratory, 1957.
- Rosenblatt, Frank. “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain.” Psychological Review, vol. 65, no. 6, 1958, pp. 386–408. https://doi.org/10.1037/h0042519
- Rosenblatt, Frank. Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Washington, DC: Spartan Books, 1962.
- Dreyfus, Hubert L., and Stuart E. Dreyfus. Making a Mind versus Modeling the Brain: Artificial Intelligence Back at a Branchpoint. New York: Free Press, 1988.
Image