Towards sustainable artificial intelligence systems: enhanced system design with machine learning based design techniques

Abstract

Efficient and timely calculations of Machine Learning (ML) algorithms are crucial for emerging technologies like autonomous driving, the Internet of Things (IoT), and edge computing. One of the primary ML techniques used in such systems is Convolutional Neural Networks (CNNs), which demand high computational resources. This requirement has led to using ML accelerators like General Purpose Graphig Processing Units (GPGPUs) to meet design constraints. However, GPGPUs have high power consumption needs, and thus, selecting the most suitable accelerator involves Design Space Exploration (DSE). This process is usually time-consuming and requires significant manual effort. This thesis presents approaches to improve the DSE process by supporting the identification of the most appropriate GPGPU for CNN inferencing systems. Different techniques are developed to quickly and precisely forecast the power consumption and performance of CNNs during inference. These approaches empower the system designer to estimate power consumption and performance for GPGPUs in the early stages of development without executing the application on real devices. Without the need to execute and profile applications on real devices, the number of prototypes can significantly be reduced. Besides the system’s power and performance requirements and the ML accelerator selection, the designer has to face the placement problem and decide whether an application is implemented on an IoT device or in the Cloud. The available network, bandwidth, and latency are crucial if the application is implemented in the Cloud. Therefore, this thesis presents a decision-supporting system that is pivotal in helping system designers Make these complex decisions. This system is designed to consider the available network, bandwidth, and latency, and it distinguishes between power or performance optimization needs, thereby empowering the system designer to make informed choices.