Hardware/Software Co-Design Towards TinyML

In the past a few years, powered by the strong need of edge intelligence, there has been an increasing interest in deploying deep neural networks on tiny hardware with limited computing power and energy (tinyML). A fundamental question needs to be addressed: given a specific edge intelligence task, what is the optimal neural architecture and the tailor-made hardware in terms of accuracy and efficiency? Earlier approaches attempted to address this question through hardware-aware neural architecture search (NAS), where fa fixed hardware design such as microcontrollers or light-weight CPUs are taken into consideration when designing neural architectures. However, we believe that the most powerful and elegant solutions should come from hardware that allows customization, such as FPGAs, ASICs, or Computing-in-Memory accelerators. For these platforms, we are the first to establish the concept of software/hardware co-design, which simultaneously explore neural architecture and the hardware design to identify the best pairs that maximize both test accuracy and hardware efficiency. In this talk, we will present the novel co-exploration frameworks for neural architecture and various hardware platforms including FPGA, ASIC and Computing-in-Memory, all of which are the first in the literature. We will demonstrate that our co-exploration concept greatly opens up the design freedom and pushes forward the Pareto frontier between hardware efficiency and test accuracy for better design tradeoffs in tinyML.