Adaptive Behavior, 1 (1)

Observations of behavior and neural activity in premotor cortex of monkeys learning to pair an arbitrary visual stimulus with one of a set of previously learned behaviors are modeled with a network comprising a large number of motor selection columns. Reinforcement learning is used to recognize new visual patterns and acquire the appropriate visual-motor conditions. The architecture employs a distributed representation in which a single pattern is coded by a small subset of columns. A column is initially able to respond to many different inputs; as it learns to trigger a motor program, its responses become more narrowly defined. Each column's output is a set of votes for the various motor programs. The votes for each program are collected by selection units, which drive a winner-take-all circuit to determine whether a particular motor program is executed. The model is successful in reproducing the sequence of behavioral responses given by the subjects, as well as a number of phenomena that have been observed at the single-unit level. Finally, we offer a comparison to the backpropagation learning algorithm that demonstrates key principles which have been designed into our algorithm.

Key Words

conditional learning; reinforcement learning; cortical columns; distributed representations, premotor cortex; primates

Pages 39-63

Postponed Conditioning: Testing a Hypothesis About Synaptic Strengthening

By Janet R. P. Halperin and David W. Dunham

Abstract

A connectionist neural circuit model of motivated behavior that uses a variant on Hebbian synaptic plasticity (Halperin, 1990, 1991) predicts that transient excitatory conditioning should occur to a stimulus presented after the US has been removed, provided that this CS presentation is postponed until just before the end of a behavioral afterdischarge. We report here some experimental confirmations of this prediction, using the social display system of male Siamese fighting fish, in which afterdischarge is visually observable. A mirror was presented and removed, to elicit social display followed by display afterdischarge. A CS object was presented just as the afterdischarge faded. After 10 to 12 such pairings, the CS shown alone elicited display. The model, however, also predicts that no excitatory conditioning should occur to a CS presented near the beginning of a long afterdischarge. This arrangement and an unpaired CS condition were used as controls, and the prediction was confirmed. However, the model further suggests that a CS presented at the beginning of an afterdischarge should show excitatory conditioning if the display afterdischarge could be artificially shortened so that the CS presentation would come just before the end of the afterdischarge. In a second experiment, the CS was presented immediately after mirror removal, and then afterdischarge was ended by feeding the fighting fish a guppy. Since the CS now predicted the arrival of food as well as the end of the afterdischarge, we depressed conditioned feeding responses by food-satiating the fish before presenting the CS alone. Display conditioning was confirmed. Controls were included for the effects of feeding.

Key Words

connectionism; synaptic strengthening; conditioning; learning; backward conditioning; aggression; motivation; Betta splendens

Pages 65-90

The Evolution of Strategies for Multiagent Environments

By John J. Grefenstette

Abstract

SAMUEL is an experimental learning system that uses genetic algorithms and other learning methods to evolve reactive decision rules from simulations of multiagent environments. The basic approach is to explore a range of behavior within a simulation model, using feedback to adapt its decision strategies over time. One of the main themes in this research is that the learning system should be able to take advantage of existing knowledge where available. This has led to the adoption of rule representations that ease the expression of existing knowledge. A second theme is that adaptation can be driven by competition among knowledge structures. Competition is applied at two levels in SAMUEL. Within a strategy composed of decision rules, rules compete with one another to influence the behavior of the system. At a higher level of granularity, entire strategies compete with one another, driven by a genetic algorithm. This article focuses on recent elaborations of the agent model of SAMUEL that are specifically designed to respond to multiple external agents. Experimental results are presented that illustrate the behavior of SAMUEL on two multiagent predator-prey tasks.

Key Words

genetic algorithms; sequential decision problems

Pages 91-122

Evolving Dynamical Neural Networks for Adaptive Behavior

By Randall D. Beer and John C. Gallagher

Abstract

We would like the behavior of the artificial agents that we construct to be as well-adapted to their environments as natural animals are to theirs. Unfortunately, designing controllers with these properties is a very difficult task. In this article, we demonstrate that continuous-time recurrent neural networks are a viable mechanism for adaptive agent control and that the genetic algorithm can be used to evolve effective neural controllers. A significant advantage of this approach is that one need specify only a measure of an agent's overall performance rather than the precise motor output trajectories by which it is achieved. By manipulating the performance evaluation, one can place selective pressure on the development of controllers with desired properties. Several novel controllers have been evolved, including a chemotaxis controller that switches between different strategies depending on environmental conditions, and a locomotion controller that takes advantage of sensory feedback if available but that can operate in its absence if necessary.

Key Words

neural networks; genetic algorithms; motor control; chemotaxis; locomotion

back to TOC, back to top

Adaptive Behavior, 1 (1)

Adaptive Behavior

Volume 1, Number 1

Summer 1992

Table of Contents

Andrew H. Fagg and Michael A. Arbib

A Model of Primate Visual-Motor Conditional Learning