Neurons with graded response have collective computational properties like those of two-state neurons

Neurons with graded response have collective computational properties like those of two-state neurons

May 1984 | J. J. Hopfield
J. J. Hopfield proposed a model of a large network of neurons with graded responses (sigmoid input-output relations) that exhibits collective computational properties similar to those of two-state neurons. This deterministic model closely matches the earlier stochastic model based on McCulloch-Pitts neurons. The model retains the ability to perform content-addressable memory (CAM) and other emergent collective properties. The presence of these properties in more biologically accurate neurons supports the idea that such properties are used in biological systems. The model's collective states correspond simply to those of the original model, and its connection to graded response systems is established. Equations that include the effect of action potentials in the graded response system are also developed. The original model used two-state threshold neurons with outputs of 0 or 1. Real neurons have continuous input-output relations and integrative time delays due to capacitance, which should be represented by differential equations. The original model used a stochastic algorithm with sudden 0-1 or 1-0 state changes. This paper shows that the important properties of the original model remain intact when these two simplifications are eliminated. Although the properties of the new continuous neurons may not yet closely match those of real neurons, a major conceptual obstacle has been eliminated. A CAM constructed on the original model's ideas using operational amplifiers and resistors will function. A continuous, deterministic model was constructed that retains all significant behaviors of the original model. The output of each neuron is a continuous and monotone-increasing function of the input. The model's equations represent a network of electrical amplifiers and have been shown to function as a CAM. The model's energy function ensures convergence to stable states, and its deterministic nature allows it to be used in computational tasks requiring an energy function. The stable states of the two models have a simple correspondence, and the continuous model retains the same flow properties as the stochastic model. The continuous model has the same flow properties in its continuous space as the stochastic model in its discrete space. It can be used in CAM or other computational tasks requiring an energy function. The qualitative effects of antisymmetric parts of T_ij should have similar effects on CAM operation in both models. The new computational behaviors, such as learning sequences, produced by antisymmetric contributions to T_ij in the stochastic model will also hold for the deterministic continuous model. The stable states of the continuous system have a simple correspondence with those of the stochastic system. For symmetric T, the continuous system has an energy function that corresponds to the stochastic system's energy function. The energy function ensures convergence to stable states, and the continuous model's stable states correspond to those of the stochastic model. The continuous model's energy function has a quadratic form, leading to a simple correspondence between the two models. The continuous model's stable states can be fewer than those of the stochastic model with the same T matrix, but the existing stable states correspond to those of the stochastic model. The continuous model supplements the original stochastic description.J. J. Hopfield proposed a model of a large network of neurons with graded responses (sigmoid input-output relations) that exhibits collective computational properties similar to those of two-state neurons. This deterministic model closely matches the earlier stochastic model based on McCulloch-Pitts neurons. The model retains the ability to perform content-addressable memory (CAM) and other emergent collective properties. The presence of these properties in more biologically accurate neurons supports the idea that such properties are used in biological systems. The model's collective states correspond simply to those of the original model, and its connection to graded response systems is established. Equations that include the effect of action potentials in the graded response system are also developed. The original model used two-state threshold neurons with outputs of 0 or 1. Real neurons have continuous input-output relations and integrative time delays due to capacitance, which should be represented by differential equations. The original model used a stochastic algorithm with sudden 0-1 or 1-0 state changes. This paper shows that the important properties of the original model remain intact when these two simplifications are eliminated. Although the properties of the new continuous neurons may not yet closely match those of real neurons, a major conceptual obstacle has been eliminated. A CAM constructed on the original model's ideas using operational amplifiers and resistors will function. A continuous, deterministic model was constructed that retains all significant behaviors of the original model. The output of each neuron is a continuous and monotone-increasing function of the input. The model's equations represent a network of electrical amplifiers and have been shown to function as a CAM. The model's energy function ensures convergence to stable states, and its deterministic nature allows it to be used in computational tasks requiring an energy function. The stable states of the two models have a simple correspondence, and the continuous model retains the same flow properties as the stochastic model. The continuous model has the same flow properties in its continuous space as the stochastic model in its discrete space. It can be used in CAM or other computational tasks requiring an energy function. The qualitative effects of antisymmetric parts of T_ij should have similar effects on CAM operation in both models. The new computational behaviors, such as learning sequences, produced by antisymmetric contributions to T_ij in the stochastic model will also hold for the deterministic continuous model. The stable states of the continuous system have a simple correspondence with those of the stochastic system. For symmetric T, the continuous system has an energy function that corresponds to the stochastic system's energy function. The energy function ensures convergence to stable states, and the continuous model's stable states correspond to those of the stochastic model. The continuous model's energy function has a quadratic form, leading to a simple correspondence between the two models. The continuous model's stable states can be fewer than those of the stochastic model with the same T matrix, but the existing stable states correspond to those of the stochastic model. The continuous model supplements the original stochastic description.
Reach us at info@study.space