Energy Efficient Algorithms for Real Time Data Processing in Reconfigurable Computing Environments

Abstract

With exponential demand for edge computing coming in computer vision 
driven applications such as autonomous vehicles, smart devices, among others, energy efficiency has become a major problem. This typically means that 
excessive power is used by traditional video processing methods which often 
involve reading and processing redundant data, especially in higher frame 
rate situations. Here, we investigate recent efforts that integrate reconfigurable hardware and intelligent algorithms to achieve orders of magnitude 
reduction in energy, while preserving real-time performance, for the most 
pervasive visual tasks. In this paper, we study a new system, which combines 
a lightweight pixel masking algorithm with a reconfigurable CMOS image 
sensor, allowing the selective omission of uneventful regions in the readout phase. This hardware–algorithm co–design framework maximizes energy 
savings for applications including autonomous driving and augmented reality 
while optimizing both front end sensor operations and back end neural network processing.