Frese, Udo
Loading...
Preferred name
Frese, Udo
Official Name
Frese, Udo
Email
ufrese@informatik.uni-bremen.de
ORCID
GND
7 results
Now showing 1 - 7 of 7
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Uncertainty driven pose estimation - for rigid objects in CNN-based pipelines(2026-04-10); ; ; Nowadays, image-based object recognition and pose estimation are highly active research areas due to their importance in robotic perception and interaction. While modern CNN-based pose estimators achieve great results, they lack transparency regarding the trustworthiness and precision of individual estimates. This lack of certainty inhibits further processing of the results and deters deployment in production environments due to reliability concerns. As an answer, this thesis proposes a fusion-based approach in which, due to a novel output architecture, the CNN self-estimates the amount of information obtained, resulting in individual 6D uncertainty estimates per 6D pose estimate. Specifically, the CNN predicts the observed object points pixel-wise, along with the precision in the image plane of those predictions. All such gathered perspective information is then fused (without linearization) into a single, globally valid 13 × 13-sized information matrix, which is then regressed to yield the six-dimensional result. This separation allows the CNN to operate solely in image space, whereas the conversion from 2D image space to 6D pose is solved analytically. Additionally, the intermediate result of the globally valid information matrix facilitates the fusion with auxiliary information, such as depth, stereo, and prior knowledge, with ease, as it is simply a 13 × 13 matrix addition. With this approach, the pose is regressed from a fusion of all available data, unlike the more ad hoc approach of combining estimates in postprocessing. Also, the CNN call is wholly unaffected by the addition of these supplemental data. An extensive evaluation of the proposed architecture on multiple benchmark datasets showcases meaningful uncertainty estimates while maintaining competitive pose performance. Also, it shows that adding auxiliary information can significantly improve pose performance, but always relative to the amount of new information gained while maintaining the quality of the estimated uncertainty.doctoral thesis18 41 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Underwater Visual Multi-Modal 3D Sensing(2020-04-23); ; ; In the underwater domain, optical sensors are extremely limited with respect to range, resolution, and accuracy in comparison to most terrestrial remote sensors. The reason for this is the medium water, which heavily interacts with electromagnetic signals and therefore reduces their corresponding signal-to-noise ratio. Also, many underwater areas can only be visited by remotely operated vehicles due to water pressure, turbidity, and or strong currents, posing a high risk for humans. This combination considerably increases the complexity of underwater metrology, and many applications currently require highly skilled personnel and large support vessels. Here, a simplification of these applications is presently effectively prevented by the performance gap of underwater optical systems in comparison to their terrestrial counterparts. Motivated by the above limitations, this research work evaluates different optical sensing modalities when applied to the underwater domain and identifies their possible sweet spots. Based on these considerations, several novel fusion strategies for passive-active optical systems are presented able to reconstruct whole underwater scenes with high accuracy without relying on additional navigation systems. For their evaluation, a novel self-referenced optical 3D underwater scanner is implemented and used for several test setups as well as real-world scenarios. The implementation also includes a novel method for in-air calibration of flat-port cameras and integration into bundle adjustment frameworks for visual pose estimation. Here, the evaluation demonstrates that passive-active optical systems outperform standard methods when underwater sensor motion is a critical design parameter. The most significant advantage of self-referenced optical 3D scanners is that they compensate sensor motion in the same sensor domain as 3D measurements take place. This reduces the complexity of sensor co-calibration, ensures a similar accuracy for sensor pose and scene depth estimation, and broadens their possible application to smaller sensor platforms.doctoral thesis385 279 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Real-time ship recognition and georeferencing for the improvement of maritime situational awareness(2024-08-09); ; ; In an era where maritime infrastructures are paramount to societies, the need for advanced maritime situational awareness solutions has become increasingly important. Existing ship monitoring procedures, such as the Automatic Identification System (AIS), have limitations, suffer from delayed updates, and are vulnerable to cyberattacks. Other technologies, such as satellite imagery and radar, face challenges in real-time applications due to delays in acquiring and processing data. The use of optical camera systems and image processing can improve situational awareness, allowing real-time usage of maritime infrastructure footage. However, the number of video streams available poses a challenge for maritime operators, who could benefit from summarized spatial information of recognized ships, irrespective of their size and type, presented on a map in real-time. This motivates the development of automated ship recognition and georeferencing technologies. Moreover, the deployment of such camera systems, equipped with embedded devices, allows for local data processing on the edge to minimize network demand, reduce energy usage, decrease latency, cut costs, and enhance data protection. This thesis, integrating six of my publications, presents a comprehensive investigation into leveraging deep learning and computer vision to advance real-time ship recognition and georeferencing for the improvement of maritime situational awareness. I present a novel dataset for ship recognition and georeferencing, ShipSG, which facilitates the development and validation of recognition and georeferencing methodologies. The dataset contains 3,505 images and 11,625 ship masks with their corresponding class, geographic position, and length. Through a series of studies of state-of-the-art deep-learning-based object recognition algorithms, I introduce a custom real-time segmentation architecture, ScatYOLOv8+CBAM. This architecture was created and optimized for the NVIDIA Jetson AGX Xavier as an embedded system. ScatYOLOv8+CBAM incorporates the 2D scattering transform, a novel addition that enhances YOLOv8 in real-world applications such as ship segmentation. Additionally, the performance is further improved with the integration of attention mechanisms. The proposed architecture exceeds state-of-the-art methods by more than 5%, achieving a mean Average Precision (mAP) of 75.46%. The inference speed, once the customized architecture is deployed on the embedded system using TensorRT, is 25.3 ms per frame. Furthermore, I address the need for precision in recognizing small and distant ships and their real-time processing of full-resolution images on embedded systems with an enhanced slicing mechanism that performs batch inference and merges predictions, achieving mAP improvements ranging from 8% to 11%. The recognized ships are georeferenced using my proposed method, which automatically calculates the georeferencing pixel of the recognized ships and uses homographies to provide the geographic position of ships from single images without prior camera knowledge. In the quantitative analysis, the georeferencing method achieved a positioning error of 18 m ± 10 m for ranges inside the port basin (up to 400 m) and 44 m ± 27 m outside (from 400 m to 1200 m). The main findings reveal significant advancements in maritime situational awareness with the practical demonstration of the applicability of the methodologies in real-world scenarios, such as the detection of abnormal ship behavior, camera integrity assessment, and 3D reconstruction. The approach not only outperforms existing methods in terms of accuracy and processing speed but also provides a framework for seamlessly integrating recognized and georeferenced ships into real-time systems, enhancing operational effectiveness and decision-making for maritime authorities. The integration of these methodologies into embedded systems represents a pivotal advancement in the domain, offering a scalable and efficient solution for improving maritime situational awareness and response capabilities. This thesis contributes to the maritime computer vision field by establishing a benchmark for ship segmentation and georeferencing research, demonstrating the viability of deep-learning-based recognition and georeferencing methods for real-time maritime monitoring.doctoral thesis446 143 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Adaptive Degrees of Freedom for Shared Control of Assistive Robots(2024-10-02); ; ; Robots have become a prominent part of our daily lives, be it as a versatile tool at work or as an obedient household helper. This is especially exciting for people with physical limitations, since designated assistive robots harbour a large potential to enhance their users' autonomy and quality of life. Following this line of thought, the field of assistive robotics introduces mechanical assistants to people who would otherwise struggle with activities of daily living. However, this necessitates adequate and potentially personalised control methods. Focusing on wheelchair-mounted robotic arms, this thesis discusses currently available options, evaluates directly applicable manual alternatives, and proposes a novel shared control based on adaptive degrees of freedom (DoFs). With a participatory design, each element is developed and evaluated in close collaboration with the target group, thus allowing for appropriate integrations and realistic assessments. For the contemporary manual analysis, users evaluated the manufacturer-provided input device in comparison to a gamepad, 3D mouse, and a command-based voice control. Here, the participants expressed an eagerness for personalised adaptability, as well as an explicit willingness to train in the use of more complex but capable systems, such as a 3D mouse. Heeding this, this thesis introduces the novel shared control approach of Adaptive DoFs: A camera-based sensor system probabilistically analyses the current situation to generate the most likely directions of robot motion and maps these to the user's input device, effectively replacing the classically available cardinal DoF. For the user, this feels like the system anticipating their next move and providing support without taking over control. An implementation of this proposed adaptive control was thoroughly evaluated with the target group, using e.g. smart glasses, showcasing high user acceptance with a steep learning curve and high success rates.doctoral thesis257 298 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, State Estimation Solely Based on Prior Knowledge and Inertial Sensors(2022-11-14); ; ; How do we localize ourselves? Ever since GPS exists, it is common to know where we are and how to get to our desired location. Unfortunately, GPS is unavailable indoors. Scientists are looking for an alternative technology that can fill this localization gap. One approach is to fuse knowledge about our environment and our movement measured with inertial sensors. A particular difficulty of these sensors is that their pose (position and orientation) estimation error grows over time. This so-called drift can lead to false estimations such as passing through a wall. These false estimations could be corrected by using prior knowledge of the wall’s location. In this work, I investigate how prior knowledge can be fused with inertial sensor measurements. The practical aim of this thesis is to eliminate the drift without additional sensors. I investigate three types of prior knowledge regarding the environment and the movement: The human gait pattern, terrain maps, and event-domain maps. For all three types, I follow the concept of modeling the prior knowledge as probability distributions of the system’s state. This modeling enables the usage of standard probability-based algorithms to estimate the position and orientation and to fuse the knowledge with sensor measurements. The human gait is an alternating pattern of stance and swing phases. I show a new approach based on the Interacting Multiple Model Filter that can detect the phase and improve the velocity estimate of the inertial sensor. The approach automatically detects whether the sensor measurements match the probability distribution of the stance or swing phase. Simultaneously, it corrects the measurement errors of the inertial sensor by taking into account the probability distributions. The evaluation shows the potential of this method, albeit further development is required to outperform state of the art approaches. Terrain maps define the height of a vehicle or a human given its position in the horizontal plane. This can be modeled as a so-called pseudo measurement. We act like there is a sensor that measures the height above the surface but always returns zero since there is no height difference. In this way, a probability distribution is modeled that constrains the position to the surface. I investigate terrain maps with the practical example of track cycling. I show that terrain maps can yield full observability of the position and orientation; in other words, that they are able to correct the growing error of the inertial sensor. Thereby, only the curved parts of the track yield information about the position. As a result, the position can be tracked during 10km drives with an error of 1:08m (RMSE). Event-domain maps are a particular type of maps that specify where activities can be performed. For example, it is only possible to climb stairs at staircases. I investigate this type of knowledge at bouldering, where the climbers grip the holds of a route. The map represents a probability distribution of possible grip positions. I develop a two-step method where the first step estimates the transition between two holds. In a second step, the transitions are refined using the event-domain map. The estimated error improves from 0:266m (median) to 0:132m compared to an integrating solution without a map. Overall, modeling the three types of prior knowledge successfully reduces the drift in all cases. The human gait pattern can be utilized with a new kind of state estimator, which needs further investigation. The map-based types of knowledge correct the drift of the inertial sensor in the experiments. For the terrain map, it is even possible to prove the correction mathematically. This shows that prior knowledge modeled as prior distribution is effective to estimate the position solely with inertial sensors.doctoral thesis339 395 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, A novel three-finger dexterous hand with visual-based grasp planning and tactile-based stable grasping(2023-12-19); ; ; ; Robotic manipulation is a complex field that still faces numerous challenges. With the development of collaborative robots, the end-effector area is just beginning to gain traction, and electrically driven end-effectors are just starting to be widely used in the industry. One of the fundamental requirements for a robot to achieve practical applications is stable grasping. However, in reality, most stable grasping relies on specific static conditions and human experience. As the robotic industry continues to develop more complex and diverse applications, the need for stable grasping to support these high-level applications increases. To achieve more complex and stable grasping functions, a complex end-effector, such as a dexterous hand, becomes essential hardware. This thesis focuses on building a novel practical dexterous hand that can be used in robotic manipulation research. To make better use of the dexterous hand, visual-based grasp planning, and tactile-based stable grasping are necessary to form a stable grasping system. In this thesis, three crucial topics were selected and divided into three parts of the work: dexterous hand, grasp planning, and stable grasping. The dexterous hand part includes the main design work of a modular three-finger dexterous hand called DoraHand and partial work on another one called Eagle Shoal. The performance of the DoraHand and the tactile sensor module is showcased through experiments. Two-finger and five-finger versions of the DoraHand have also been developed and tested in real applications, providing a reliable hardware foundation for further research. The grasp planning part focuses on providing a grasp planning solution for the dexterous hand. As an essential function of using an end-effector, this part starts with an analytic solution that considers the limitations of the dexterous hand mechanism and grasp quality evaluation. A grasp planning network has been developed using both analytic and data-driven approaches. The network features a multi-finger grasp plan representation method and has been successfully verified. The stable grasping part is the final application of this thesis, where the hardware provides the foundation and the stable grasping algorithm utilizes the tactile sensor. An open-source visual-tactile dataset has been developed using the Eagle Shoal dexterous hand. The stable grasping algorithm, built based on this dataset, has been successfully verified with different types of end-effectors, including DoraHand and suction cup gripper. Overall, these three parts of work constitute the critical components of a stable grasping system using a dexterous hand. This system and related dataset enable further research in stable grasping and robotic manipulation. The primary objective of this thesis has been successfully achieved with the development of DoraHand, which has been used by over twenty research institutes and companies. The algorithms developed for grasp planning and stable grasping serve as a foundation for future research in this field, while the dataset can be used as a benchmark for comprehensive robotic research. Further development is needed to explore the potential applications of the dexterous hand in robotic manipulation.doctoral thesis384 513 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Interconnection between communication and suboptimality for distributed control systems(2021-03-18); ; ; Distributed systems are present in every day's life: For example, distributed systems occur in the form of the internet, system architectures utilising edge computing, robotic scenarios in manufacturing processes, or the upcoming spreading of autonomous, connected vehicles. Distributed systems require a coordination or communication mechanism and an appropriate choice of control methods. Here, the Distributed Model Predictive Control scheme was applied in various application fields such as process control, robotic scenarios and traffic scenarios. As these robotic or traffic scenarios as examples for distributed systems encounter a high changing dynamic, the chosen coordination and communication mechanism should be efficient due to necessary synchronisation between the subsystems. A non-cooperative setting is examined, where each subsystem follows an individual target and has to exchange information steadily to ensure first and foremost collision avoidance. With utilisation of wireless communication resources and their finite bandwidth, this work present methods to attenuate the communication effort between the subsystems. Based on a quantisation implemented on the communication exchange between the subsystems, it is shown that each subsystems achieves their assigned target, i.e. the overall system still converge. The improvements for the communication load is presented and for a robotic scenario convergence is shown utilising a weaker assumption than terminal constraints. Additionally, to improve convergence, dynamic priority rules are introduced, which calculate an dynamic optimisation order for the subsystems, where different schemes are utilised: While the first is based on a simple sort based on a priority criterion in every time instant, the latter prevents deadlocks and examine additionally concurrent execution of the subsystems. We close with an overview of possible applications and transformation to other scenarios and give an outlook on further developments.doctoral thesis646 840