On 2019 July 19th, CLOTHILDE team received the visit of Dr. Eren Erdal Aksoy from Halmstad University, Sweden.
 
Dr. Aksoy offered a research seminar to IRI:
Title: A Holistic View on Manipulation Semantics: From Robot Perception to Execution
Description: Robots require a generic action representation in order to recognize, learn, and imitate observed tasks without any human intervention. Defining such a representation is a challenging problem due to the high inter-individual variability that emerges during the execution of actions. Conventional methods approach this problem by either considering continuous trajectory profiles or employing predefined symbolic action knowledge. The main challenge, however, still remains in linking perceived continuous sensory signals to discrete symbolic object or action concepts.
In this talk, I will promote a new holistic view on manipulation semantics, which combines the perception and execution of manipulation actions in one unique framework, so-called “Semantic Event Chain” (SEC). The SEC concept is an implicit spatiotemporal formulation that encodes actions by coupling the observed effect with the exhibited roles of manipulated objects. In the first part of the talk, I will explain how such a semantic action encoding can allow robots to link continuous visual sensory signals (e.g. image sequences) to their symbolic descriptions (e.g. action primitives). I will then elaborate on creating a robot-agnostic semantic library of actions to be further employed to generate complex chained manipulation sequences while grounding high-level symbolic plans into the low-level sensory-motor. To highlight the scalability of manipulation semantics, I will finally introduce various applications of SECs on learning object affordances, coupling language and vision, and memorizing episodic experiences.