Principles of Neural Network Architecture Design - Invertibility and Domain Knowledge
File | Description | Size | Format | |
---|---|---|---|---|
00108536-1.pdf | 6.91 MB | Adobe PDF | View/Open |
Other Titles: | Architekturprinzipien zum Design Neuronaler Netzwerke - Invertierbarkeit und Domain-Wissen | Authors: | Behrmann, Jens ![]() |
Supervisor: | Maaß, Peter ![]() |
1. Expert: | Maaß, Peter ![]() |
Experts: | Fischer, Asja | Abstract: | Neural networks architectures allow a tremendous variety of design choices. In this work, we study two principles underlying these architectures: First, the design and application of invertible neural networks (INNs). Second, the incorporation of domain knowledge into neural network architectures. After introducing the mathematical foundations of deep learning, we address the invertibility of standard feedforward neural networks from a mathematical perspective. These results serve as a motivation for our proposed invertible residual networks (i-ResNets). This architecture class is then studied in two scenarios: First, we propose ways to use i-ResNets as a normalizing flow and demonstrate the applicability for high-dimensional generative modeling. Second, we study the excessive invariance of common deep image classifiers and discuss consequences for adversarial robustness. We finish with a study of convolutional neural networks for tumor classification based on imaging mass spectrometry (IMS) data. For this application, we propose an adapted architecture guided by our knowledge of the domain of IMS data and show its superior performance on two challenging tumor classification datasets. |
Keywords: | Deep Learning; Invertible Neural Networks; Adversarial Examples; Imaging Mass Spectrometry; Normalizing Flows | Issue Date: | 30-Oct-2019 | Type: | Dissertation | Secondary publication: | no | URN: | urn:nbn:de:gbv:46-00108536-13 | Institution: | Universität Bremen | Faculty: | Fachbereich 03: Mathematik/Informatik (FB 03) |
Appears in Collections: | Dissertationen |
Page view(s)
912
checked on Apr 2, 2025
Download(s)
750
checked on Apr 2, 2025
Google ScholarTM
Check
Items in Media are protected by copyright, with all rights reserved, unless otherwise indicated.