Towards self-organising Action Selection

Abstract

Systems with multiple parallel goals (e.g. autonomous mobile robots) have a problem analogous to that of action selection in ethology. Architectures such as the subsumption architecture (Brooks) involve multiple sensing-and-acting agents within a single robot on its own if allowed. Which to give control at a given moment is normally regarded as a (difficult) problem of design. In a quest for a scheme where the agents decide for themselves in a sensible manner, I introduce a model where the agents are not only autonomous but are in full competition with each other for control of the robot. Interesting robots are ones where no agent achieves total victory, but rather a serires of compromises are reached. Having the agents operate by the reinforcement learning algorithm Q-learning (Watkins) allows the introduction of an algorithm called `W-learning', by which the agents learn to focus their competitive efforts in a manner similar to agents with limited spending power in an economy. In this way, the population of agents organises its own action selection in a coherent way that supports parallelism and opportunism. In the empirical section, I show how the relative influence an agent has on its robot may be controlled by adjusting its rewards. The possibility of automated search of agent-combinations is considered.