Gaze data handler for a brain-machine interface framework


The project should realize a new module for the UBiCI software framework of the Neuroinformatics Group. UBiCI is an extensive, highly modular framework to design and implement online brain-machine interface (BMI) applications. It is written in C++ and depends on Qt, RSB and OpenCV. Currently, it comprises modules for EEG and ECG data acquisition, pre-­‐processing, feature extraction, classification, device control (e.g., sending commands to a robot) as well as an interface for graphical programming and configuration of BMI experiments. UBiCI is fully open source, but currently outside the CITEC available only upon request. As a prerequisite for the proposed project, UBiCI will be moved to the CITEC open source server prior to the start of the working period.

Current CITEC/ AG NI research projects (specifically: IP 19) require an extension of UBiCI by a module that handles eye tracking (ET) data alongside the other modalities. In particular, the implementation of online fixation and saccade detection, as well as of a so-called smooth pursuit based calibration procedure are of high relevance. This can be handled in a generic fashion, independent of particular ET hardware, because these algorithms process the raw gaze position coordinates, which are delivered by any eye tracker. The new module will serve the development of novel BMIs based on so-­called fixation-related potentials (FRP). FRPs are neural activity that is time-­‐locked to fixation onsets. Thus, their application in a BMI requires the online detection of the fixations to provide valid time stamps for appropriately segmenting the EEG data for further processing.

In the context of the call, the implementation of the aforementioned module is particularly feasible, because: (1) No hardware is required, ET data will be provided to simulate the online case for implementation and testing. (2) The algorithms can be implemented by using standard C++ functions; no additional dependencies (beside those already mentioned) are needed. (3) Example code (not part of UBiCI) for understanding the necessary steps for fixation and saccade detection is available in the AG NI.

The benefit for the applicant is to gain insight into the field of BMI research beyond the specific task required in the project. He/she will have the chance to learn about the different methods and the steps needed to realize an own BMI system.


Resources and Working Environment

The applicant will only need to check out UBiCI and RSB from the respective repositories and install the additional dependencies (all open source). Implementation and testing should be done fully off-­‐ site, using a standard Linux system and QtCreator. We will provide ET data for testing. Upon the successful completion of the project, the new module will be added to UBiCI.

Roadmap and Open Access

The applicant will be supported via virtual meetings (Skype) on a regular basis during the working period (e.g. twice a week). We will discuss the progress of the project, the so far obtained results and eventually solve any issue that may arise.

M1 Getting to know UBiCI, its basic structure, and how to compile and install it.
M2 Working version of UBiCI installed.  
M3 Implementing core version of algorithms and testing with ET data.
M4 Algorithms are working as expected.
M5 Integrating core algorithms into UBiCI as a new module (LibUBiCI Eyetracking).
M6 UBiCI successfully builds on CI Server and passes the tests.