Embedded neuromorphic attention model leveraging a novel low-power heterogeneous platform

Description

Neuromorphic computing has been identified as an ideal candidate to exploit the potential of event-based cameras, a promising sensor for embedded computer vision. However, state-of-the-art neuromorphic models try to maximize the model performance on large platforms rather than a trade-off between memory requirements and performance. We present the first deployment of an embedded neuromorphic algorithm on Kraken, a low-power RISC-V-based SoC prototype including a neuromorphic spiking neural network (SNN) accelerator. In addition, the model employed in this paper was designed to achieve visual attention detection on event data while minimizing the neuronal populations' size and the inference latency. Experimental results show that it is possible to achieve saliency detection in event data with a delay of 32ms, maintains classification accuracy of 84.51% and consumes only 3.85mJ per second of processed input data, achieving all of this while processing input data 10 times faster than real-time. This trade-off between decision latency, power consumption, accuracy, and run time significantly outperforms those achieved by previous implementations on CPU and neuromorphic hardware.

Abstract

This work was supported by the European Union's ERA-NET CHIST-ERA 2018 research and innovation programme under grant agreement ANR-19-CHR3-0008 and SNF 20CH21 186991. It was partially supported by the SNF Innosuisse 103.364 IP-ICT. The authors are grateful to the OPAL infrastructure from Université Côte d'Azur for providing resources and support.

Abstract

International audience

Additional details