One of the visions in AI based robotics are household robots that can autonomously handle a variety of meal preparation tasks. Based on this scenario, we present a best practice tutorial on how to create actionable knowledge graphs that a robot can use for execution of task variations of cutting actions. We implemented a solution for this task that integrates all necessary software components in the framework of the robot control process. In the context of this tutorial, we focus on knowledge acquisition, knowledge representation and reasoning, and simulating robot action execution, bringing these components together into a learning environment that – in the extended version – introduces the whole control process of Cognitive Robotics. In particular, the Tutorial will detail necessary concepts a knowledge graph should include for robot action execution, how web knowledge can be automatically acquired for the domain of cutting fruits, and how the created knowledge graph can be used to let robots execute tasks like slicing a cucumber or quartering an apple. The learning environment follows an immersive approach, using a physics-based simulation environment for visualization purposes that helps to illustrate the concepts taught in the tutorial.
The tutorial consists of two parts:
- Extraction of Relevant Action and Object Knowledge from the Web presented by Jan-Philipp Töberg
- Accessing Web Knowledge & Knowledge Graphs for Action Parameterisation presented by Dr. Michaela Kümpel
The tutorial is held at the Extended Semantic Web Conference 2024 and the International Conference on Hybrid Human-Artificial Intelligence. More information on the topics can be found on our website.