A Multiplierless Architecture for Image Convolution in Memory

Abstract

Image convolution is a commonly required task in machine vision and Convolution Neural Networks (CNNs). Due to the large data movement required, image convolution can benefit greatly from in-memory computing. However, image convolution is very computationally intensive, requiring ${(n-(k-1))}^2$ Inner Product (IP) computations for convolution of a $n\times n$ image with a $k\times k$ kernel. For example, for a convolution of a 224 × 224 image with a 3 × 3 kernel, 49,284 IPs need to be computed, where each IP requires nine multiplications and eight additions. This is a major hurdle for in-memory implementation because in-memory adders and multipliers are extremely slow compared to CMOS multipliers. In this work, we revive an old technique called ‘Distributed Arithmetic’ and judiciously apply it to perform image convolution in memory without area-intensive hard-wired multipliers. Distributed arithmetic performs multiplication using shift-and-add operations, and they are implemented using CMOS circuits in the periphery of ReRAM memory. Compared to Google’s TPU, our in-memory architecture requires 56× less energy while incurring 24× more latency for convolution of a 224 × 224 image with a 3 × 3 filter.