Machine Learning Techniques for Autonomous Multi-Sensor Long-Range Environmental Perception System

An environment perception system is one of the most critical components of an automated vehicle, which is defined as a vehicle where the driver does not require to monitor the vehicle’s behavior and its surroundings during driving. This thesis addresses some of the main challenges in the development of vision-based environment perception methods for automated driving, focusing on railway vehicles. The thesis aims at developing methods for detecting obstacles on the rail tracks in front of a moving train to reduce the number of collisions between trains and various obstacles, thus increasing the safety of rail transport.

In the field of autonomous obstacle detection for automated driving, besides recognising the objects on the way, the crucial information for collision avoidance is estimated distances between the vehicle and the recognised objects (e.g. cars, pedestrians, cyclists). With the limited capabilities of current state-of-the-art sensor-based environment perception approaches, it is unrealistic to detect distant objects and estimates the distance to them. Mid-to-long-range obstacle detection system is one of the fundamental requirements for heavy vehicles such as railway vehicles or trucks, due to required long braking distance. However, this problem is unaddressed in the computer vision community. The emphasis of this thesis is on the development of robust and reliable algorithms for real-time vision-based mid-to-long-range obstacle detection. In this thesis, the algorithms for obstacle detection from single cameras were developed and evaluated on images captured from RGB, Thermal and Night-Vision Cameras.

The developed algorithms are based on advanced machine/deep learning techniques. The development of machine-learning-based algorithms was supported by a novel mid-to-long-range obstacle detection dataset for railways that is proposed in the thesis, which compiles annotated images with the object class, bounding box, and ground truth distance to the object.

The developed novel methods for autonomous long-range obstacle detection, tracking, and distance estimation for railways were evaluated on real-world images, which were recorded in different illumination and weather conditions by the obstacle detection system mounted on a static test-bed set-up on the straight rail track and as well on a moving train. Although the focus is on railways, the developed algorithms are also capable to use for road vehicles, hence evaluated on the images of road-scene captured by a camera mounted on moving cars.