3D RSBag Visualizer for Multimodal Data

English

RSBag visualizer is a web-based visualization tool designed for display of multimodal time-series and objects provided in an RSB-compliant format. The main purpose of the visualizer is the online/offline display of the experimentally and otherwise generated multimodal data in a easily interpretable form of a 3d environment.

In this project the existing framework previously used for visualization of haptic and vicon data, should be extended to display a wide range of new spatio-temporal features and scene components. The resulting software may be of interest to anybody working with analysis, generation and validation of multimodal data.

 

Visualizer

The current visualizing system consists of a server and a web-based client. Both, the multimodal time-series as well as the scene objects are provided to the server in an RSB-compliant format.

Necessary skills and requirements

The participant should have very good JavaScript and Python skills. Knowledge of Common Lisp is a plus.
Progress of the project should be measured by a weekly report by the participant by email. The participant should create a weekly plan for the period of the project beforehand.
The results of the project should be published on https://opensource.cit-ec.de/.

Roadmap

The extension of the present visualizer (see Figure 1) consists of introduction of new spatio-temporal features:

  • sub-window displays: 2D feature plot, video-based feature plot (see Figure 2)
  • visualization of temporal segments
  • visualization of linked structures
  • augmentation of a scene with new 3D objects
  • augmentation of a scene with a display of dynamic/static user-defined object features

To enable the participant of MORe to experiment with the above-stated features, we will provide all the necessary experimental data.

Figure 1 Figure 2