Neuromorphic vision using spiky neural network for object recognition

Abstract

Spiking Neural Networks (SNNs) offer a promising alternative by leveraging event-driven processing, which reduces power consumption while maintaining high detection accuracy. However, training SNNs remains a challenge due to the absence of conventional backpropagation and the need for specialized training algorithms. This study develops a neuromorphic vision system using Spiking Neural Networks (SNNs) for energy-efficient, low-latency object recognition, inspired by the human visual system. Leveraging event-based datasets—DVS Gesture (dynamic gestures) and N-Caltech101 (static objects)—the study implements a hardware-agnostic SNN framework to outperform traditional Convolutional Neural Networks (CNNs) in real-time applications like robotics and autonomous vehicles. Using Python libraries (Norse, Brian2), the system processes asynchronous event streams, reducing computational overhead compared to frame-based methods. A React.js frontend, combined with D3.js, enables real-time visualization of spike events and detection outputs. Key objectives include designing an SNN model, preprocessing event data, benchmarking against CNNs (e.g., MobileNetV2) on accuracy, latency, and energy efficiency, and addressing software-based neuromorphic challenges. Implemented on Google Colab’s T4 GPU, the SNN achieved 60% mAP on N-Caltech101 and ~85% accuracy on DVS Gesture, with significantly lower latency (30ms vs. 60ms) and energy consumption (~10x less) than CNNs. Despite challenges like data scarcity and SNN training complexity, the system demonstrates the viability of software-based neuromorphic vision, contributing to accessible, brain-inspired AI for resource-constrained environments.