Comparing photorealism in game engines for synthetic maritime computer vision datasets

Abstract

Computer vision for real-world applications faces data acquisition challenges, including accessibil- ity, high costs, difficulty in obtaining diversity in scenarios or environmental conditions. Synthetic data usage has surged as a solution to these obstacles. Leveraging game engines for synthetic dataset creation effectively enriches training datasets with increased diversity and richness. The choice of game engine, pivotal for generating photorealistic simulations, significantly influences synthetic data quality. This thesis aims to compare Unreal Engine’s and Unity Engine’s capabilities in generating synthetic maritime datasets to support ship recognition applications. To this end, the real-world maritime dataset ShipSG has been replicated in the corresponding game engines to recreate the same scenarios. To this end, a true-to-scale 3D model of the Doppelschleuse was crafted using Blender, aiming to reproduce the scenery of the ShipSG images. This model is further used in the corresponding game engines, along with 3D models of different vessels, to recreate these scenarios in rendered synthetic datasets. The performance of the generated synthetic datasets is benchmarked against the real-world ShipSG dataset using the YOLOv8 model for ship segmentation. The comparison includes an assessment of performance differences between manually annotated synthetic datasets and those with auto-generated annotations by Unity Engine. Additionally, the comparison investigates ef- fects, such as sunlight presence and lens distortion, to find the configuration that most enhances performance when using YOLOv8. The dataset generated using Unity engine with manual annota- tions, both with and without lens distortion, provided the best accuracy in ship recognition (mAP of 72.3%). Both were used to augment the ShipSG dataset to train YOLOv8. The configuration with lens distortion provides the highest mAP increase, of 0.4% compared with YOLOv8 perfor- mance on ShipSG when no synthetic data is used (76.5% vs 76.9%). This evidence underscores that utilizing game engines can effectively support and enhance ship recognition tasks. This thesis demonstrates the potential of synthetic datasets to augment real-world maritime data, highlights the importance of detailed and accurate 3D modelling, and systematically provides a methodology to select the most fitting game engine for the generation of photorealistic images to enhance maritime computer vision applications.