Memory and Learning


Current smart home solutions do not have the capability to memorize user idiosyncrasies or let alone interaction events, leading to repeating tedious interactions with the user. We therefore target to provide a functionality for developing a memory that provides an intelligent interface for applications that need history information for learning, interaction or other purposes.

Formalising and modelling human machine interaction in smart environments

Human machine interaction plays a crucial role in smart environments such as smart homes. In order to analyse humans and create artificial behaviours and actions in this domain, one needs to be able to appropriately describe, store, and query interaction data. This especially gains high importance when the complexity of systems rises as embodied agents are introduced as interaction partners (Köster et al. 2015). For example, an agent in an interactive domain might detect a human entering it. Upon detection and identification, it needs to check if this person is known and would therefore internally state a query such as presented in Listing 1.

Listing 1: Query formatted in Cypher notation targeting the question: “Has     this specific person, i.e. Marlena in the example query, been seen before and what were interaction partners and topics of these interactions?”

As a foundation to this important part of the CSRA large scale project we presented an ontology for modelling human machine interaction in smart environments to support querying interaction data at the conceptual level, abstracting from specific and heterogeneous data schemata (Köster et al. 2016). Existing ontologies (such as the Semantic Sensor Network Ontology or the Time Ontology) have been reused to compose this conceptualisation of the domain. We validated our approach along our formulated competency questions as well as an experimental setup to to store data in the graph database Neo4j (c.f. Figure 1).


Figure 1: Example data in the Neo4j database used to validate our underlying schema.











A Domain-Specific Query Language for Interaction Data

Writing queries within a complex domains is an error-prone and time consuming task. Developers need to be rather well informed and domain experts and understand the data structure and formulate effective queries. In current research a model-driven approach is followed in which different components of the data management system are generated automatically. This allows us to provide integrated tools such as a Query-Designer (c.f. Figure 2) which allows users to write Cypher queries against the domain of embodied interaction in smart environments.


Figure 2: The Query-Designer providides support for advanced query development and execution.





It extends the Cypher syntax and language with several features and provides explicit domain knowledge, code completion, syntax checking and direct in-tool query execution. A currently running study is designed to evaluate this approach and the tool usability.

Data reduction

Given the large number of sensors and actors, a smart robotic environment can produce huge amounts of data. It is therefore important to reduce the amount of information for storage and learning by intelligent methods of feature reduction.

Efficient nonlinear dimensionality reduction for high-dimensional sensor signals


Sensor data which are gathered 24/7 from heterogeneous sensors cause high dimensional, distributed, and possibly defective signals; as a promising first step towards an efficient representation, we have investigated the potential of modern non-linear dimensionality reduction methodology. Besides its capability of inferring information-preserving low-dimensional representations, we have demonstrated its ability to help in an unsupervised integration of distributed sensor signals by exploring their topological constraints, and its ability to automatically correct sensor faults by a reference to sensor redundancy (Mokbel, Schulz 2015).