The cognitive structure and control of full body movements - experimental analysis and computational modelling

Acronym: 
COBOMOV
Term: 
2008-05 till 2012-10
Research Areas: 
A
D
Abstract: 

The aim of this project is to investigate complex movements of the human body in sports, dance or every-day life activities, and to understand how these movements are controlled, learned and reproduced under changing conditions. Based on experimental results, we want to develop a simulation framework that integrates different control systems and their interaction, including a high level control network that generates and adapts sequences of motor primitives representing control on the level of mental representation.

Methods and Research Questions: 

Generating cognitive behaviour of virtual agents and robots requires more than a simple, reflex driven sensory-motor mapping. It requires hierarchical, internal models of the outside world that can help the agent to anticipate, to plan ahead and to adapt its motor primitives to changes in the environment. We want to develop a computational model that simulates human whole-body movements by integrating different control systems and their interactions. An important aspect in the control of complex movements is the level of mental representations in long term memory. We regard mental representations in long term memory as being built upon a hierarchical structure of basic action concepts (BACs) that represent the major building blocks of actions on the level of mental representation. Embedded into a hierarchical basic concept system, they bind together the functional and motor features of an action and the sensory characteristics that are perceived during movement execution. The goal of the computational modelling part of this project is to develop a simulation framework that allows the integration and the combination of promising concepts developed in the recent years, like echo state networks, parametric bias recurrent networks, and powerful training methods, like AMALGAM and EVOLINO. On top of this framework, a high level control network will be added that generates and adapts sequences of motor primitives to form chains of BAC’s, thus representing control on the level of mental representation.

Outcomes: 

In the project COBOMOV, we study recurrent neural networks for adaptive behaviour generation. It was shown that a special class of networks called Echo State Networks (ESN) can easily store multiple motor patterns as attractors in a single network and generate novel patterns by combining and blending already learned patterns using bifurcation inputs. Using Multiobjective Optimization of structural ESN parameters including full-weight optimization, the model capacity of ESN’s was analysed. A modular architecture was designed that couples multiple ESNs and a hierarchical self-organizing map (HSOM). The HSOM implements cognitive structures of basic action concepts to provide input- and reference values for the ESNs. It can integrate perceptual features of the environment, proprioceptive sensory data of a robot body and higher level commands (intention, affordance) to select a proper motor program.

Publications: