Intermittent-Aware Neural Architecture Search

Abstract

Intermittently executing deep neural network (DNN) inference powered by ambient energy, paves the way for sustainable and intelligent edge applications. Neural architecture search (NAS) has achieved great success in automatically finding highly accurate networks with low latency. However, we observe that NAS attempts to improve inference latency by primarily maximizing data reuse, but the derived solutions when deployed on intermittent systems may be inefficient, such that the inference may not satisfy an end-to-end latency requirement and, more seriously, they may be unsafe given an insufficient energy budget. This work proposes iNAS, which introduces intermittent execution behavior into NAS. In order to generate accurate neural networks and corresponding intermittent execution designs that are safe and efficient, iNAS finds the right balance between data reuse and the costs related to progress preservation and recovery, while ensuring the power-cycle energy budget is not exceeded. The solutions found by iNAS and an existing HW-NAS were evaluated on a Texas Instruments device under intermittent power, across different datasets, energy budgets and latency requirements. Experimental results show that in all cases the iNAS solutions safely meet the latency requirements, and substantially improve the end-to-end inference latency compared to the HW-NAS solutions.