Motion Tracking based teleoperation of a virtual human in MORSE
The Modular OpenRobots Simulation Engine (MORSE), originally developed by LAAS-CNRS, is a Blender-based versatile simulator for robotics. MORSE is an generic simulator for academic robotics. It focuses on realistic 3D simulation of small to large environments, indoor or outdoor, with one to tenths of autonomous robots. MORSE can be entirely controlled from the command-line. Simulation scenes are generated from simple Python scripts. The simulator comes with a set of standard sensors (cameras, laser scanner, GPS, odometry,...), actuators (speed controllers, high-level waypoints controllers, generic joint controllers) and robotic bases (quadrotors, ATRV, Pioneer3DX, generic 4 wheel vehicle, PR2,...). New ones can easily be added.
MORSE rendering is based on the Blender Game Engine. The OpenGL-based Game Engine supports shaders, provides advanced lightning options, supports multi-texturing, and use the state-of-the-art Bullet library for physics simulation.
MORSE already provides a human avatar that is capable of interacting with its virtual environment (and of course robots) by, i.e., picking up and moving objects in the scene. In this project you will develop a stand-alone software component that makes use of current motion/skeleton tracking hardware and available frameworks (Kinect2, ASUS Xtion, etc.). This external component will communicate with MORSE (Socket, ROS, RSB, PyMorse, your choice) in order to control/steer the human avatar.
It is a bonus if applicants already have access to a Asus Xtion, Kinect, or Kinect2 camera. However,
the MORe initiative will ensure access to such a device.
The project is required to run on LINUX.
Required Skills
- In general you will need experience with the Kinect (or similar) camera and a corresponding software framework in order to receive skeleton tracking data.
- For the implementation of a skeleton tracking component C/C++ is preferred. Thus, experience in C/C++ is advantageous. However, if you prefer any other major programming language you arewelcome to present us your skills.
- Basic Blender knowledge is a plus but not required, in case MORSE blender models need to be adjusted (can be learned during the project).
- For the MORSE integration you will need at least a basic PYTHON knowledge.
- Having experience with GIT is a plus.
Roadmap
- Get the hardware running on a LINUX distribution, using the tracking framework of choice.
- Generation of a generic/extensible architecture for skeletal motion tracking devices and data.
- Implementation of an easy to use and middleware-enabled motion tracking software for HRI.
- Integration into MORSE.
- Testing and deployment strategies.
In order to organize the software development process we will use either github or our own open source server (git-based). The frequency of meetings via Google Hangout (preferred) or Skype will be discussed in the bonding phase. Besides video conferencing, instant messaging and email is always an option.