Hybrid Deep Learning: how combining data-driven and model-based approaches solves inverse problems in computed tomography and beyond

Abstract

Artificial neural networks from the field of deep learning are increasingly becoming the state of the art in more and more applications. Their success is based on learning complex relationships in a system purely from data. For this, the data-driven networks often require hundreds of thousands of reference examples. They are contrasted by model-based approaches that use mathematical methods to describe the processes in a system. They work without large amounts of data but often cannot cover all the nuances of an application.

In inverse problems, model-based approaches have been the standard so far. Here, the necessary amount of data to use purely data-driven deep learning is usually unavailable. In addition, requirements are placed on the model properties that cannot always be proven for classical neural networks. Hybrid deep learning models that combine data-driven and model-based approaches can solve these challenges. In recent years, their research has steadily gained importance.

In this thesis, several hybrid deep learning approaches for solving inverse problems are presented and further developed. These include the deep image prior (DIP) and conditional invertible neural networks (CINN). The reconstruction problem in computed tomography (CT) serves as a central example to compare the models with each other, as well as to reveal their strengths and weaknesses. This is done in particular concerning the unique challenges in inverse problems, such as lack of data and ill-posedness. For this purpose, a realistic medical CT dataset is presented and used. The performed comparison for medical and industrial data clearly shows that the hybrid approaches are superior to the classical, model-based methods in many areas. Countless applications from inverse problems can thus already benefit from hybrid deep learning approaches.