Evolutionary neural architecture search for the task of road traffic prediction.

Abstract

The topic of this dissertation is the application of evolutionary neural architecture search to find suitable neural networks for predicting speed and flow from road traffic data. The thesis begins by describing the measurement data and describing the forecasting problem. Following this, fundamental concepts in the fields of Machine Learning, Deep Learning, and Neural Architecture Search (NAS), particularly concerning application, are explained. The last part of this dissertation consists of five articles to which the author of this thesis has made a significant contribution. The first two articles provide an overview of the problem of traffic data prediction, concerning measurement data from the city of Bremen. The machine learning model k-nearest neighbors is introduced and applied to the measurement data. In addition, we evaluate data imputation methods to improve models. In the third article, we compare combined polynomial regression models, a simple machine learning model, with graph convolutional neural networks. These are neural networks that include special opera- tions incorporating spatial dependencies between measurement points. Our evolutionary neural architecture search framework is presented in the fourth article. The outcome of the genetic algorithm used in our framework depends on the fitness, i.e. performance on the dataset, of each architecture in the search space. While the choice of validation loss as fitness is ideal w.r.t. the accuracy, it slows down the algorithm tremendously since it necessitates training the neural networks until convergence. Hence, to make usage of our framework viable, in the fifth article, we evaluate zero-cost proxies, which compute a fitness for architectures based on singular forward or backward passes through the network. Therefore, evaluating network fitness only takes a few compared to multiple hours. We show that the naswot zero-cost proxy is robust w.r.t. random initializations of weights, network sizes and batch sizes and has a high spearman rank correlation with the validation loss. My contribution is a neural architecture search framework that finds neural network architectures that are especially powerful for predicting road traffic data. My NAS framework finds an architecture for a given dataset that can keep up with or outperform handcrafted neural networks and neural networks found by other NAS frameworks in terms of performance and computation time.