Evolutionary neural architecture search for the task of road traffic prediction.

The topic of this dissertation is the application of evolutionary neural architecture search
to find suitable neural networks for predicting speed and flow from road traffic data.
The thesis begins by describing the measurement data and describing the forecasting
problem. Following this, fundamental concepts in the fields of Machine Learning, Deep
Learning, and Neural Architecture Search (NAS), particularly concerning application,
are explained. The last part of this dissertation consists of five articles to which the
author of this thesis has made a significant contribution.
The first two articles provide an overview of the problem of traffic data prediction,
concerning measurement data from the city of Bremen. The machine learning model
k-nearest neighbors is introduced and applied to the measurement data. In addition, we
evaluate data imputation methods to improve models. In the third article, we compare
combined polynomial regression models, a simple machine learning model, with graph
convolutional neural networks. These are neural networks that include special opera-
tions incorporating spatial dependencies between measurement points. Our evolutionary
neural architecture search framework is presented in the fourth article. The outcome of
the genetic algorithm used in our framework depends on the fitness, i.e. performance on
the dataset, of each architecture in the search space. While the choice of validation loss
as fitness is ideal w.r.t. the accuracy, it slows down the algorithm tremendously since
it necessitates training the neural networks until convergence. Hence, to make usage of
our framework viable, in the fifth article, we evaluate zero-cost proxies, which compute
a fitness for architectures based on singular forward or backward passes through the
network. Therefore, evaluating network fitness only takes a few compared to multiple
hours. We show that the naswot zero-cost proxy is robust w.r.t. random initializations
of weights, network sizes and batch sizes and has a high spearman rank correlation with
the validation loss.
My contribution is a neural architecture search framework that finds neural network
architectures that are especially powerful for predicting road traffic data. My NAS
framework finds an architecture for a given dataset that can keep up with or outperform
handcrafted neural networks and neural networks found by other NAS frameworks in
terms of performance and computation time.