The Argus FPGA-Based CNN Accelerator Generator

Abstract

Convolutional neural networks (CNNs) are used to solve a wide range of challenging computer vision problems. However, the power of CNNs comes at the cost of large amounts of computations. High-end FPGAs are loaded with compute resources and are widely deployed in data centers. As such, accelerating CNNs with FPGAs is a promising solution for meeting the computation demand of CNNs in data centers.

The main obstacle in using FPGAs as CNN accelerators is that FPGA programming requires hardware design knowledge that most CNN developers and users do not have. To solve this problem, we designed Argus, an FPGA-based CNN accelerator generator. Given the information of a CNN and FPGA board, Argus automatically generates a highly optimized accelerator design for running the CNN on the FPGA board, thus enabling users with no hardware design knowledge to effectively use FPGAs for CNN acceleration.

This dissertation presents four important components of Argus's design. The first one is the multi-CLP CNN accelerator paradigm, which spreads the layers of a CNN to multiple heterogeneous CNN layer processors (CLPs), with each CLP specialized for a subset of the CNN's layers. The resulting multi-CLP accelerator can achieve close to 100% dynamic utilization of arithmetic units. The second one is Escher, which is a CLP design with flexible data buffering to deal with CNN layers that are bottlenecked by weight transfers. Then there is Medusa, a resource-efficient and high-performance memory interconnect specialized to meet the data transfer requirements of multi-CLP accelerators. Last but not least, there is Cocktail, which is a scalable and extensible multi-CLP optimizer that enables the future extension of Argus to support more recent CNN variants and more varied CLP designs in a multi-CLP accelerator.