This paper discusses connectionist learning procedures, which are a type of neural network model that emphasizes computational power rather than biological accuracy. The goal of these procedures is to enable networks to learn internal representations of their environment. The learning process involves adjusting the strengths of connections between units to minimize error in the network's output. Several gradient-descent procedures have been developed, which are effective for small tasks but need improvement for larger, more complex tasks.
Connectionist models consist of many simple processing units connected by weighted links. Each unit's state is determined by the input it receives. The state of a unit is typically a nonlinear function of its total input. The paper explores three main research areas: search, representation, and learning. Learning is the focus, but an introduction to search and representation is necessary for understanding the purpose of learning.
For search, the network can find a solution by iteratively updating the states of the units until they reach a stable state. For representation, the paper discusses the difference between local and distributed representations. Distributed representations are more efficient and damage-resistant, and can lead to better generalization. However, they can make it difficult to represent multiple things at once.
In learning, the paper discusses different procedures: supervised, reinforcement, and unsupervised. Supervised learning requires a teacher to specify the desired output, while reinforcement learning only requires a single scalar evaluation of the output. Unsupervised learning constructs internal models that capture regularities in input vectors without additional information.
The paper also discusses associative memories without hidden units, which can store associations between input and output vectors. However, they have limitations when the input vectors are not orthogonal. Nonlinear associative nets can perform better in such cases.
The paper then discusses backpropagation, a multi-layer least squares procedure that is effective for networks with hidden units. It is used to discover semantic features in data, map text to speech, and recognize phonemes. The paper highlights the importance of error surfaces and the challenges of finding the global minimum in these surfaces. It also discusses the use of postprocessing to improve the output of backpropagation networks.This paper discusses connectionist learning procedures, which are a type of neural network model that emphasizes computational power rather than biological accuracy. The goal of these procedures is to enable networks to learn internal representations of their environment. The learning process involves adjusting the strengths of connections between units to minimize error in the network's output. Several gradient-descent procedures have been developed, which are effective for small tasks but need improvement for larger, more complex tasks.
Connectionist models consist of many simple processing units connected by weighted links. Each unit's state is determined by the input it receives. The state of a unit is typically a nonlinear function of its total input. The paper explores three main research areas: search, representation, and learning. Learning is the focus, but an introduction to search and representation is necessary for understanding the purpose of learning.
For search, the network can find a solution by iteratively updating the states of the units until they reach a stable state. For representation, the paper discusses the difference between local and distributed representations. Distributed representations are more efficient and damage-resistant, and can lead to better generalization. However, they can make it difficult to represent multiple things at once.
In learning, the paper discusses different procedures: supervised, reinforcement, and unsupervised. Supervised learning requires a teacher to specify the desired output, while reinforcement learning only requires a single scalar evaluation of the output. Unsupervised learning constructs internal models that capture regularities in input vectors without additional information.
The paper also discusses associative memories without hidden units, which can store associations between input and output vectors. However, they have limitations when the input vectors are not orthogonal. Nonlinear associative nets can perform better in such cases.
The paper then discusses backpropagation, a multi-layer least squares procedure that is effective for networks with hidden units. It is used to discover semantic features in data, map text to speech, and recognize phonemes. The paper highlights the importance of error surfaces and the challenges of finding the global minimum in these surfaces. It also discusses the use of postprocessing to improve the output of backpropagation networks.