Adaptive Behavior, 1 (4)

Several researchers of animal behavior, such as Tinbergen and Baerends, have proposed hierarchical mechanisms for action selection. Maes, among others, has argued against mechanisms of this type because of the top-down control and lack of robustness inherent in these hierarchical decision structures, or rigid switching mechanisms, in which decisions are made at every level. Two alternatives exist to these hierarchical decision structures: Maes and others have designed non-hierarchical action selection mechanisms, and Rosenblatt and Payton have outlined a hierarchical mechanism that does not make decisions at every level. In this article, use of the two types of hierarchies for action selection is discussed, and the Rosenblatt and Payton approach--with free flow of information, combination of evidence, and the ability to select compromise candidates--is supported. It is also argued that the problem of action selection is, by nature, intrinsically hierarchical, and so Rosenblatt and Payton-like hierarchies (free-flow hierarchies) are more suitable for action selection than non-hierarchical mechanisms. Evidence from observation of animal behavior and from computer simulation testing is used to lend support to these claims.

Key Words

action selection; behavioral choice; hierarchies; mechanisms; switching; animal behavior

Pages 421-436

On Rate-Sensitive Habituation

By John E. R. Staddon

Abstract

The responses elicited by many stimuli diminish in strength with repeated stimulus presentation (habituation). This property is consistent with models in which short-term memory for recent stimuli suppresses the current response. Habituation also depends on stimulus spacing: It occurs more rapidly when interstimulus intervals (ISIs) are short than when they are long, but also recovers more rapidly after short ISIs (rate sensitivity). The effect of ISI on habituation rate is consistent with a simple one-stage process, but the effect of ISI on recovery rate seems to require a serial process in which two or more habituating units are cascaded, with earlier (peripheral) units in the series having shorter time constants than later (central) units. Rate-sensitive habituation may underlie puzzling effects of reinforcement learning such as the partial-reinforcement and successive-contrast effects.

Key Words

integrator; rate sensitivity; interstimulus interval; partial reinforcement; contrast effect; cascade; cascaded systems

Pages 437-454

Efficient Learning and Planning Within the Dyna Framework

By Jing Peng and Ronald J. Williams

Abstract

Sutton's Dyna framework provides a novel and computationally appealing way to integrate learning, planning, and reacting in autonomous agents. Examined here is a class of strategies designed to enhance the learning and planning power of Dyna systems by increasing their computational efficiency. The benefit of using these strategies is demonstrated on some simple abstract learning tasks.

Key Words

reinforcement learning; dynamic programming; sequential decision problems

Pages 455-468

Leg Coordination Mechanisms in the Stick Insect Applied to Hexapod Robot Locomotion

By Kenneth S. Espenschied, Roger D. Quinn, Hillel J. Chiel, Randall D. Beer

Abstract

Three of the mechanisms believed to be responsible for leg coordination in the stick insect, Carausius morosus, are used to control the straight-line locomotion of a hexapod robot on a smooth surface. The robot walks with a continuum of statically stable insectlike gaits in response to a single, scalar user input that controls the speed of locomotion. This control strategy is highly robust in the sense that the controller continues to maintain its basic function of causing the robot to walk effectively despite large perturbations to the controller. This controller robustness was demonstrated by inducing a wide variety of lesions (severing of connections) and performing parameter sensitivity studies.

Key Words

hexapod robot; coordination mechanisms; stick insect; robust control; locomotion

Pages 469-472

Author Index to Volume 1

Pages 473-478

Key Word Index to Volume 1

back to TOC,back to top

Adaptive Behavior, 1 (4)

Adaptive Behavior

Volume 1, Number 4

Spring 1993

Table of Contents

Toby Tyrrell

The Use of Hierarchies for Action Selection