1. Bringing up robot: Fundamental mechanisms for creating a self-motivated, self-organizing architecture

Doug Blank, Deepak Kumar, Lisa Meeden, and Jim Marshall. Cybernetics and Systems, 2005, 36(2)

Here is a  PDF of the full paper.

1.1. Introduction

• Most AI researchers imagine the task that they want a robot to accomplish in terms of their own actions, concepts, desires, and motivations
• Task-oriented design
• Top-down (armchair introspection)
• Bottom-up (evolutionary computation)
• The task itself is the problem - inherent anthropomorphic bias

1.2. Overview of a Developmental Robotics Paradigm

Goals:

• Turn on the robot, and let it begin experiencing the world
• What type of learning system?
• Creates a representation that allows for generalization
• Not necessarily "one-shot learning"; not overly considered with speed of learning
• Not necessarily limited to biological plausibility, but developmental efficacy
• Self-organizing, self-ratcheting intrinsic mechanism
• No particular task, per se
• Develop an intertwined, emergent system of abstractions, anticipations, and self-motivations
• Not sure how useful such a system will be to us
• May be unable to make it do things that it doesn't want to do
• May never develop the symbolic level

Issues:

• May take a long time
• If there is no task, what will be our measures of success?
• May not end up with a system that is useful (to us)
• Could end up with a system that is just as complicated as the one that we are attempting to "model"
• May help to shed light on consciousness and emergent intelligence

1.3. The Intrinsic Developmental Algorithm

1.3.1. Discovering Abstractions

Self-organizing Map (see SOMExampleProgram for example)

1.3.2. Anticipating the Future

Simple Recurrent Network

1.3.3. Motivation

Like a so-called co-evolutionary arms-race

1.4. Experiment #1: Using abstractions to govern neural network learning

Problem: a neural network on a robot does not get symbols spoon-fed to it. Rather, the world is coming in through its sensors all the time. There may be long periods in which the input doesn't change. If the network indiscriminately attempts to learn on every time step, then this can lead to catastrophic forgetting. In such a situation, the network doesn't learn to generalize at all.

Solution: balance the input/target training evenly over all types of patterns.

The Neural Network Governor: rather than constantly learning on the current input/target pair, create a SOM-like filter that creates abstractions, categorizes them, and trains in a balanced way.

1.5. Experiment #2: Using abstractions to create purposeful behaviors