Lab 2: Interactive Activation Network

This lab is gratefully taken from Axel Cleeremans (Original page)

Consider the relationships captured by the following table:

This can be represented in an Interactive Activation Network as shown below for a subset of the data. Each person has a "person" node as well as a node for each feature (name, profession, age, etc). Links between nodes within each feature set are inhibitory. Links form a person to his own features are excitatory.

Armed with that, now have a look at the applet below.

 
Source code
Built with Processing

This web page contains an applet written in Java. In the present computing landscape that may or may not work. It probably requires an uptodate version of Java. If it doesn't work in the first browser you try, try another. Chrome will probably not display the applet at all (thanks Chrome). Firefox, Safari and Internet Explorerare may work, but they all have their quirks. If none of them work, go to Axel Cleeremans Home pagea> and download the standalone application for Mac or for Windows. If that doesn't work, we may be out of options.

This applet is a demonstration of "Interactive Activation and Competition" neural networks, as first described by J.L. McClelland in the following paper:

McClelland, J.L. (1981). Retrieving general and specific information from stored knowledge of specifics. Proceedings of the Third Annual Meeting of the Cognitive Science Society, 170-172.

A more complete description of the network is also available in Rumelhart, D.E., McClelland, J.L., and the PDP research group (1986).Parallel Distributed Processing Volume 1: Foundations, MIT Press.

The model was a landmark illustration of how simple networks of interconnected elements (artificial neurons, so to speak) can be organized so as to form a memory system that exhibits several crucial properties of human memory, including

  1. content adressability: The fact that we can easily retrieve an entire memory based on any of its elements (i.e., hearing the name of a friend brings his face to one's mind, and a host of other facts such as when we last met him)
  2. graceful degradation: The fact that our performance does not degrade catastrophically in the presence of lesions or noise, but rather in a gradual, progressive manner (think of how your understanding of a foreign language changes as ambient noise increases)
  3. default assignment: The fact that we can usually make educated guesses about missing information on the basis of similarity (e.g., if I tell you that Brutus is an animal that has four legs and wags his tail when it sees me, you'll rightfully guess that Brutus is most likely a dog even though you don't know this for a fact).
  4. spontaneous generalization: The fact that we automatically form representations of the features that are shared by the exemplars of a category when presented with the category. This is the basis for the emergence of social stereotypes, and for prototype formation in general.

In the demo above and in the original article, the model captures what someone might know about characters inspired from the 1961 musical "West Side Story", in which two gangs (the "Jets" and the "Sharks") fight for territory and overall dominance. Units, the activation level of which represents the fact that the feature they stand for is true to some degree, are organized in separate groups, each corresponding to the different possible values of a single feature, such as the name of an individual, his age, his education level, his marital status, his occupation, or the gang to which he belongs.

All units within a group are connected to each other by inhibitory connections, so that when one becomes active it will tend to inhibit the others, thus reflecting the fact that one cannot be both 30 years of age and 40 years of age, for instance. All the units corresponding to the features that characterize a single individual are linked through excitatory connections to a single unit (see the two inner circles) that binds this individual's various features together, thus representing the model's memory of that individual.

The simple system nevertheless exhibits complex dynamics of interaction and competition as activation spreads through the network in response to your input. To explore the model, simply click on any unit to activate it, and watch the model progressively settle in a state in which only a few units - those that represent features that are most compatible with your input - remain active. Reset the model to its default state by pressing "r". Pause and resume the simulation by pressing SPACE.