Adaptive Behavior, 2 (3)

This article explores the use of a real-valued modular genetic algorithm to evolve continuous-time recurrent neural networks capable of sequential behavior and learning. We evolve networks that can generate a fixed sequence of outputs in response to an external trigger occurring at varying intervals of time. We also evolve networks that can learn to generate one of a set of possible sequences based on reinforcement from the environment. Finally, we utilize concepts from dynamical systems theory to understand the operation of some of these evolved networks. A novel feature of our approach is that we assume neither an a priori discretization of states or time nor an a priori learning algorithm that explicitly modifies network parameters during learning. Rather, we merely expose dynamical neural networks to tasks that require sequential behavior and learning and allow the genetic algorithm to evolve network dynamics capable of accomplishing these tasks.

Key Words

neural networks; genetic algorithms; sequential behavior; reinforcement learning

Pages 247-275

Training Agents to Perform Sequential Behavior

By Marco Colombetti and Marco Dorigo

Abstract

This article is concerned with training an agent to perform sequential behavior. In previous work, we have been applying reinforcement learning techniques to control a reactive agent. Obviously, a purely reactive system is limited in the kind of interactions it can learn. In particular, it can learn what we can pseudosequences--that is, sequences of actions in which each action is selected on the basis of current sensory stimuli. It cannot learn proper sequences, in which actions must be selected also on the basis of some internal state. Moreover, it is a result of our research that effective learning of proper sequences is improved by letting the agent and the trainer communicate. First, we consider trainer-to-agent communication, introducing the concept of reinforcement sensor, which lets the learning robot explicitly know whether the last reinforcement was a reward or a punishment. We also show how the use of this sensor makes error recovery rules emerge. Then we introduce agent-to-trainer communication, which is used to disambiguate ambiguous training situations--that is, situations in which the observation of the agent's behavior does not provide the trainer with enough information to decide whether the agent's move is right or wrong. We also show an alternative solution to the problem of ambiguous situations, which involves learning to coordinate behavior in a simpler, unambiguous setting and then transferring what has been learned to a more complex situation. All the design choices we make are discussed and compared by means of experiments in a simulated world.

Key Words

classifier systems; genetic algorithms; training; sequential behavior; autonomous agents

Pages 277-305

Using Genetic Algorithms to Learn Reactive Control Parameters for Autonomous Robotic Navigation

By Ashwin Ram, Ronald Arkin, Gary Boone, Michael Pearce

Abstract

This article explores the application of genetic algorithms to the learning of local robot navigation behaviors for reactive control systems. Our approach evolves reactive control systems in various environments, thus creating sets of "ecological niches" that can be used in similar environments. The use of genetic algorithms as an unsupervised learning method for a reactive control architecture greatly reduces the effort required to configure a navigation system. Unlike standard genetic algorithms, our method uses a floating point gene representation. The system is fully implemented and has been evaluated through extensive computer simulations of robot navigation through various types of environments.

Key Words

reactive control; genetic algorithms; robot navigation; machine learning

back to TOC, back to top

Adaptive Behavior, 2 (3)

Adaptive Behavior

Volume 2, Number 3

Winter 1994

Table of Contents

Brian M. Yamauchi and Randall D. Beer

Sequential Behavior and Learning in Evolved Dynamical Neural Networks