Institute for Artificial Intelligence

Übergeordnete Einrichtung

Fachbereich 03: Mathematik/Informatik (FB 03)

Country

City

Bremen

Gerade angezeigt 1 - 10 von 14

Some of the metrics are blocked by your
consent settings
Towards Reactive Robotics with a Pinch of Image-Schematic Reasoning
(RWTH Aachen, 2023)
Pomarlan, Mihai
;
Dhanabalachandran, Kaviya
;
Beetz, Michael
Today’s robots do not possess a deep understanding of interactions between physical objects that is also available to their behavior generation modules and as such show brittle performance in realistic environments. While this suggests a robot would therefore need more knowledge, its decisions should not be too complicated to arrive at or else the robot risks losing track of what matters from its environment. Thus, we investigate a mix of reactive approaches to robotics and reasoning, and propose a simplified theory of typical changes between image schemas. We show how this theory could be integrated in a robot’s perception-action loop, and describe some examples of using this theory to infer actions and perception queries for various stages of a pouring task. We are integrating this inference procedure into a simulated robot, but this integration is yet to be completed and as such, future work.
Konferenzbeitrag
34 17
Some of the metrics are blocked by your
consent settings
Between Input and Output: The Importance of Modelling Transients in Meal Preparation Tasks
(RWTH Aachen, 2024)
Kümpel, Michaela
;
Hassouna, Vanessa
;
Hawkin, Alina
;
Beetz, Michael
We are moving closer to autonomous robots preparing meals. While restaurant robots in static environments already are successfully performing single actions like making pizza, the goal is to enablerobots to perform changing actions, in various environments and with any available object. Towards this goal, a methodology for creating actionable knowledge graphs that can be used to parameterise general action plans has been proposed. However, for extended failure handling towards fully automated action execution, we argue that transients need to be considered. A transient can be described as a transitory object in a task that is not the same as the input object anymore but not yet the output object of the task. For example, when pouring ingredients into a bowl to make the dough, the added ingredients form a mass of ingredients (here: a transient) that only becomes dough through mixing them. This work shows how transients can be modelled and how robots can integrate and possibly benefit from this modelling.
Konferenzbeitrag
43 17
Some of the metrics are blocked by your
consent settings
An Ontological Model of User Preferences
(RWTH Aachen, 2023)
Abdel-Keream, Mona
;
Beßler, Daniel
;
Janssen, Ayden
;
Jongebloed, Sascha
;
Nolte, Robin
The notion of preferences plays an important role in many disciplines including service robotics which is concerned with scenarios in which robots interact with humans. These interactions can be favored by robots taking human preferences into account. This raises the issue of how preferences should be represented to support such preference-aware decision making. Several formal accounts for a notion of preferences exist. However, these approaches fall short on defining the nature and structure of the options that a robot has in a given situation. In this work, we thus investigate a formal model of preferences where options are non-atomic entities that are defined by the complex situations they bring about.
Konferenzbeitrag
20 28
Some of the metrics are blocked by your
consent settings
Transforming Web Knowledge into Actionable Knowledge Graphs for Robot Manipulation Tasks
(RWTH Aachen, 2024)
Beetz, Michael
;
Cimiano, Philipp
;
Kümpel, Michaela
;
Motta, Enrico
;
Tiddi, Ilaria
One of the visions in AI based robotics are household robots that can autonomously handle a variety of meal preparation tasks. Based on this scenario, we present a best practice tutorial on how to create actionable knowledge graphs that a robot can use for execution of task variations of cutting actions. We implemented a solution for this task that integrates all necessary software components in the framework of the robot control process. In the context of this tutorial, we focus on knowledge acquisition, knowledge representation and reasoning, and simulating robot action execution, bringing these components together into a learning environment that – in the extended version – introduces the whole control process of Cognitive Robotics. In particular, the Tutorial will detail necessary concepts a knowledge graph should include for robot action execution, how web knowledge can be automatically acquired for the domain of cutting fruits, and how the created knowledge graph can be used to let robots execute tasks like slicing a cucumber or quartering an apple. The learning environment follows an immersive approach, using a physics-based simulation environment for visualization purposes that helps to illustrate the concepts taught in the tutorial.
Konferenzbeitrag
30 17
Some of the metrics are blocked by your
consent settings
Perception for imagination-enabled robots
(2025-10-14)
Mania, Patrick
;
Beetz, Michael
;
Vincze, Markus
;
Beetz, Michael
Recent advancements in robotics and computer vision have enhanced object recognition and control strategies. However, these developments do not fully tackle the challenges of autonomous manipulation in dynamic, unstructured environments like households. Current systems often rely on specialized algorithms for perception, which lack generalizability and fail to verify the plausibility of their results. This thesis proposes a comprehensive framework that enhances robotic perception and manipulation in dynamic, unstructured environments by integrating a photorealistic, physics-enabled game engine. The core contributions of this research are threefold. First, it presents a unified perception architecture that combines imagistic reasoning, process-level control, and perception task adaptation within a single system. This architecture enables robots to construct internal hypotheses, simulate expected sensor data, and verify perceptual results against rendered scenes, facilitating grounded and introspective perception in real-world tasks. Second, the thesis presents a game-engine-based belief representation, utilizing real-time simulation as an internal model of belief states to enable high-fidelity visual hypothesis generation. The simulated environment represents a dynamic world model, including the robot state, allowing the system to assess the plausibility of perceptual results and predict the visual consequences of actions. Lastly, Perception Pipeline Trees (PPTs) are introduced as a modular process model for adaptive perception execution. PPTs combine hierarchical execution with flexible control flow, supporting reactive switching, concurrent processing, and introspective verification. This model accommodates conventional vision tasks and imagistic reasoning processes within a unified representation. The framework demonstrates effectiveness in real-world applications, including household assistance scenarios where robots perform tasks such as recognizing and manipulating objects, as well as tracking and interacting with humans. By enabling robots to not only observe but also reason about their environment through simulation, this work advances task adaptability, perception accuracy, and reasoning capability, laying the foundation for the next generation of intelligent, imagination-enabled robots.
Dissertation
77 64
Some of the metrics are blocked by your
consent settings
A Plan Executive Architecture for Transferable Robot Behavior: Generalized Action Plans and Their Context-Adaptive Execution in Real-World Settings
(2025-10-02)
Kazhoyan, Gayane
;
Beetz, Michael
;
Kaelbling, Leslie Pack
;
Beetz, Michael
To enable the deployment of robots beyond constrained laboratory settings into open-world environments, such as homes, retail stores or agile factories, robot software must be capable of transferring to novel execution contexts with minimal reprogramming effort. While state-of-the-art systems demonstrate impressive capabilities in controlled settings, transferring them to new environments, applications and hardware platforms remains a labor-intensive process. This thesis addresses the challenge of transferability in autonomous mobile manipulation systems by proposing a novel plan executive architecture that enables the reuse of robot behavior specifications across diverse execution contexts. The central innovation lies in combining robot control programs written in an expressive robot programming language with a novel mixed symbolic–subsymbolic action representation, termed action designators. This pairing yields generalized action plans that effectively combine action control flow and contextual reasoning. A hierarchical generalized action plan for the mobile pick and place action category is developed as a case study. It demonstrates context-adaptive execution by dynamically grounding its action designators through context-specific action parameter inference. This inference is carried out via a modular infrastructure that allows to integrate alternative and / or complimenting parametrization engines: a geometric world-state based engine, a heuristics-based engine, an experience-based engine trained on execution logs and an observation-based engine that learns from human demonstrations in virtual reality. A rapid simulation step is used to validate inferred action parameters prior to real-world execution. The architecture further supports self-specialization by refining generalized plans via learning and template-based plan transformations, improving performance in specific contexts. The approach is implemented in a fully integrated robot system that includes motion control, perception and learning components. It is empirically validated across 40 simulated and six real-world execution contexts, involving five robot platforms and a variety of environments and applications. Experimental results support the thesis hypothesis that a single generalized plan can effectively transfer across a broad range of execution contexts with limited reprogramming. The plan executive satisfies key requirements for transferability, scalability, extensibility, reactivity, failure tolerance, self-improvement, usability and explainability, advancing the capability of autonomous robots to competently act in diverse, real-world contexts.
Dissertation
39 51
Some of the metrics are blocked by your
consent settings
Prospective perception through cognitive emulation for robot manipulation tasks: "Perceiving like humans do"
(2025-10-07)
Kenghagho Kenfack, Franklin
;
Beetz, Michael
;
Sandini, Giulio
;
Beetz, Michael
This thesis argues that bottom-up theories of perception suffer from the high semantic entropy arising from the severe spatial, temporal, and informational limitations of sensory input during everyday manipulation tasks. However, by emulating the “dark matter” of perception — including intent, functionality, utility, causality, and physis — and integrating it with sparse sensory data, robotic perception can achieve a causal, transparent, and computationally efficient ability to anticipate and explain relevant events in such tasks. To this end, the thesis introduces Probabilistic Embodied Scene Grammars (PESGs) to formalize this perceptual “dark matter.” It also presents a generator and a parser to respectively anticipate and explain event-centric scenes. The approach is demonstrated in complex real-world scenarios, including household tasks such as pancake making in kitchen environments, shopping tasks in supermarkets, and sterility testing tasks in medical laboratories.
Dissertation
106 71
Some of the metrics are blocked by your
consent settings
Towards a Knowledge Engineering Methodology for Flexible Robot Manipulation in Everyday Tasks
(RWTH Aachen, 2024)
Kümpel, Michaela
;
Töberg, Jan-Philipp
;
Hassouna, Vanessa
;
Cimiano, Philipp
;
Beetz, Michael
In the last decade, there have been great advancements in household robotics, enabling robots to autonomously accomplish household tasks. These robots are typically programmed for specific tasks and/ or objects. We hypothesise that the lack of flexibility in fulfilling new ad-hoc task requests can be overcome by a knowledge-based approach, allowing robots to infer how to address a new task or carry out known tasks on new objects. Towards this goal, we propose a knowledge-based methodology that leverages knowledge already existing on the web to construct an ontology supporting robots in reasoning about parameters that influence manipulation actions for execution of task variations on a range of objects. The ontology comprises object and action information, covering dispositions and affordances as well as task-specific properties. As a proof-of-concept, we manually construct a food-cutting ontology by importing and linking knowledge from relevant ontologies in addition to extracting and semantically enhancing knowledge from unstructured web sources. We demonstrate how robots can query the ontology and translate the contained information into action parameters. We evaluate the feasibility of the created ontology by simulating a robot accessing the ontology for parameterisation of actions to perform task variations of cutting.
text::conference output::conference proceedings::conference paper
30 15
Some of the metrics are blocked by your
consent settings
Steps Towards Generalized Manipulation Action Plans - Tackling Mixing Task
(RWTH Aachen, 2024)
Hassouna, Vanessa
;
Hawkin, Alina
;
Beetz, Michael
In the rapidly evolving field of household robotics, the ability to autonomously execute complex tasks like "Serve Me Breakfast" introduces considerable technical challenges, particularly in the precise and adaptive handling of mixing tasks that involve substances with diverse densities and viscosities. This study advances the field of cognitive robotics by delving into the intricacies of everyday, complex, and highly variable activities within household settings. These scenarios demand that robots manage multiple, interconnected actions—a concept known as schemas. We present a holistic strategy to enhance household robots with sophisticated task execution capabilities, utilizing the Cognitive Robotic Abstract Machine (CRAM) framework for strategic planning and execution. Our method begins with an in-depth analysis of human behaviour to develop a theoretical model that guides the creation of adaptable and comprehensible action plans for robots. A crucial element of our approach involves using Narrative Enabled Episodic Memories (NEEMs), which capture detailed records of task executions to aid performance analysis and experiential learning. We propose incorporating additional criteria based on events recorded in the NEEMs to assess task success. For instance, a robot maintaining a steady grip on the whisk while interacting with a fluid suggests correctly executing mixing tasks. These criteria enable further evaluation of performance through simulations, despite potential limitations in simulation fidelity. This article explores the transition from human expertise to robotic execution of mixing tasks in household environments, the methodology for gathering and analysing NEEMs, and their prospective future applications.
Konferenzbeitrag
28 14
Some of the metrics are blocked by your
consent settings
A balancing act: Ordering algorithm and image-schematic action descriptors for stacking objects by household robots
(RWTH Aachen, 2022)
Dhanabalachandran, Kaviya
;
Hedblom, Maria M.
;
Beetz, Michael
Optimising object order in stacking problems remains a hard problem for cognitive robotics research. In this paper, we continue our work on using the spatiotemporal relationships called image schemas to represent affordance spaces founded on object properties. Based on object properties, we introduce a stacking-order algorithm and describe the action descriptors using an image-schematic event segmentation format by describing a small subset using the Image Schema Logic ISL𝐹𝑂𝐿.
Konferenzbeitrag
24 18

Institute for Artificial Intelligence

Filter

Einstellungen

Sortieren nach

Ergebnisse pro Seite