Abstract

Hardware accelerators are being designed to offload compute-intensive tasks such as deep neural networks from the CPU to improve the overall performance of an application, specifically on the performance-per-watt metric. Encoder-decoderbased sequence-to-sequence models such as the Transformer model have demonstrated state-of-the-art results in end-to-end automatic speech recognition systems (ASRs). The Transformer model being intensive on memory and computation poses a challenge for an FPGA implementation. This paper proposes an end-to-end architecture to accelerate a Transformer for an ASR system. The host CPU orchestrates the computations from different encoder and decoder stages of the Transformer architecture on the designed hardware accelerator with no necessity for intervening FPGA reconfiguration. The communication latency is hidden by prefetching the weights of the next encoder/decoder block while the current block is being processed. The computation is split across both the Super Logic Regions (SLRs) of the FPGA, mitigating the inter-SLR communication. The proposed design presents an optimal latency, exploiting the available resources. The accelerator design is realized using highlevel synthesis tools and evaluated on an Alveo U-50 FPGA card. The design demonstrates an average speed-up of 32× compared to an Intel Xeon E5-2640 CPU and 8.8× compared to NVIDIA GeForce RTX 3080 Ti Graphics card for a 32-bit floating point single precision model. Index Terms—Automatic speech recognition, Hardware accelerator, Transformer, FPGAs.

Abstract

In this paper, we design and develop a hardware accelerator for computing the LU decomposition of an input matrix. Our accelerator consists of two simple linear arrays of Processing Engines (PEs), one on each of the two SLR regions of the FPGA. All the computations arising from the block LU decomposition are simplified and scheduled on these two PE arrays. On an Alveo U50 FPGA, our design achieves a peak floating-point performance of 128 GLOPS/s and an average performance of 95 GFLOPS/s. We achieve ≈ 15× speedup on latency compared to an Intel MKL implementation on a 4-core Intel Xeon CPU.

Abstract

This paper proposes a scalable distributed video analytics framework that can process thousands of video streams from sources such as CCTV cameras using semantic scene analysis. The main idea is to deploy deep learning pipelines on the fog nodes and generate semantic scene description records (SDRs) of video feeds from the associated CCTV cameras. These SDRs are transmitted to the cloud instead of video frames saving on network bandwidth. Using these SDRs stored on the cloud database, we can answer many complex queries and perform rich video analytics, within extremely low latencies. There is no need to scan and process the video streams again on a per query basis. The software architecture on the fog nodes allows for integrating new deep learning pipelines dynamically into the existing system, thereby supporting novel analytics and queries. We demonstrate the effectiveness of the system by proposing a novel distributed algorithm for real-time vehicle pursuit. The proposed algorithm involves asking multiple spatio-temporal queries in an adaptive fashion to reduce the query processing time and is robust to inaccuracies in the deployed deep learning pipelines and camera failures.

Abstract

The exponential performance growth guaranteed by Moore’s law has started to taper in recent years. At the same time, emerging applications like image processing demand heavy computational performance. These factors inevitably lead to the emergence of domain-specific accelerators (DSA) to fill the performance void left by conventional architectures. FPGAs are rapidly evolving towards becoming an alternative to custom ASICs for designing DSAs because of their low power consumption and a higher degree of parallelism. DSA design on FPGAs requires careful calibration of the FPGA compute and memory resources towards achieving optimal throughput. Hardware Descriptive Languages (HDL) like Verilog have been traditionally used to design FPGA hardware. HDLs are not geared towards any domain, and the user has to put in much effort to describe the hardware at the register transfer level. Domain Specific Languages (DSLs) and compilers have been recently used to weave together handwritten HDLs templates targeting a specific domain. Recent efforts have designed DSAs with image-processing DSLs targeting FPGAs. Image computations in the DSL are lowered to pre-existing templates or lower-level languages like HLS-C. This approach requires expensive FPGA re-flashing for every new workload. In contrast to this fixed-function hardware approach, overlays are gaining traction. Overlays are DSAs resembling a processor, which is synthesized and flashed on the FPGA once but is flexible enough to process a broad class of computations through soft reconfiguration. Less work has been reported in the context of image processing overlays. Image processing algorithms vary in size and shape, ranging from simple blurring operations to complex pyramid systems. The primary challenge in designing an image-processing overlay is maintaining flexibility in mapping different algorithms

Abstract

The popularity of High-Frequency Trading (HFT) or algorithmic trading has surged in the last ten years, largely due to the exponential increase in computing power. However, while software solutions for HFT have become increasingly optimized over time, they are still plagued by network stack, and kernel-user space separation overheads. Order book construction is a crucial step in any HFT system, as it provides a market snapshot on which trading decisions must be based and executed. This paper proposes a simple linear data structure for tracking the order book and a hybrid binary-linear search algorithm to maintain the top bid and ask offers corresponding to marketdepth, on FPGAs. Our approach takes into account that most trading activity occurs on the top of the bid and offer sides. Through design analysis and experimentation, we demonstrate that our simple approach is scalable and practical, outperforming previous methods. Index Terms—HFT, order book, FPGA

Abstract

FPGAs are increasingly significant for deploying convolutional neural network (CNN) inference models because of performance demands and power constraints in embedded and data centre applications. Object detection and classification are essential tasks in computer vision. You Only Look Once (YOLO) is a very efficient algorithm for object detection and classification with its variant Yolov3-tiny specially designed for embedded applications. This paper presents the FPGA accelerator for multiple precisions (FIXED-8, FIXED-16, FLOAT32) of YoloV3-tiny. We use a homogenous systolic array architecture with a synchronized pipeline adder tree for convolution, allowing it to be scalable for multiple variants of Yolo with a change in host driver. We evaluated the design on Terasic DE5a-Net-DDR4. The Fixed point (FP-8, FP-16) implementations attain a throughput of 57 GOPs/s (> 23%) and 46.16 GOPs/s (> 340 %). We synthesized the first FLOAT32 imnlementation attaining 11.22 GFLOPs/s.

Abstract

Image processing algorithms with intrinsic robust- ness to errors can be approximated for significant resource and energy savings while still meeting the end-user requirements. FPGA-based implementations can increase their suitability for real-time high-speed multimedia applications by leveraging Ap- proximate Computing, an independent field that explores meth- ods to reduce computation costs by allowing minor degradation in intermediate computations. With high volume of pixel level computations, corner detection algorithms such as the Harris Corner Detector (HCD), offer wide range of targets for such strategies. In this paper, we propose an hardware implementation of HCD that relies on approximating the intermediate multi- plication operations using Dynamic Range Unbiased Multiplier (DRUM). With run-time configurable bit-width control of DRUM instances, the visual quality of outputs is shown to depend on the varying accuracy of the corner response. We explore how the errors due to approximate operations propagate to the corner response and attempt to find a threshold that adapts to this inaccuracy across images. The experimental results for implementations on Virtex-7 and Zynq-7000 FPGA devices show that our approximate architecture can match the performance of other HCD implementations while hardly utilizing any on-board memory and signal processing resources. Synthesis results show that the proposed implementation achieves over 60% increase in maximum frequency compared to the base implementation. Finally, the quality metric analysis facilitates the selection of approximate configuration suited to the needs of an application. Index Terms—FPGA, Approximate Computing, DRUM, Image Processing, Harris Corner Detector, Bluespec, Zynq, Virtex

Abstract

Increasingly, pre-trained convolutional neural networks (CNNs) are being deployed for inference in various computer vision applications, both on the server-side in the data centers and at the edge. CNN inference is a very compute-intensive task. It is a challenge to meet performance metrics such as latency and throughput while optimizing power. Special-purpose ASICs and FPGAs are suitable candidates to meet these power and performance budgets simultaneously. Rapidly evolving CNN architectures involve novel convolution operations such as point convolutions, depth separable convolutions, and so on. This leads to substantial variation in the computational structure across CNNs and layers within a CNN. Because of this, FPGA reconfigurability provides an attractive tradeoff compared to ASICs. FPGA-based hardware designers address the structural variability issue by generating a network-specific accelerator for a single network or a class of networks. However, homogeneous accelerators are network agnostic and often sacrifice throughput and FPGA LUTs for flexibility. In this article, we propose an FPGA overlay for efficient processing of CNNs that can be scaled based on the available compute and memory resources of the FPGA. The overlay is configured on the fly through control words sent by the host on a per-layer basis. Unlike current overlays, our architecture exploits all forms of parallelism inside a convolution operation. A constraint system is employed at the host end to find out the per-layer configuration of the overlay that uses all forms of parallelism in the processing of the layer, resulting in the highest throughput for that layer. We studied the effectiveness of our overlay by using it to process AlexNet, VGG16, YOLO, MobileNet, and ResNet-50 CNNs targeting a Virtex7 and a bigger Ultrascale+VU9P FPGAs. The chosen CNNs have a mix of different types of convolution layers and filter sizes, presenting a good variation in model size and structure. Our accelerator reported a maximum throughput of 1,200 GOps/second on the Virtex7, an improvement of 1.2× to 5× over the recent designs. Also, the reported performance density, measured in giga operations per second per KLUT, is 1.3× to 4× improvement over existing works. Similar speed-up and performance density is also observed for the Ultrascale+VU9P FPGA.

Abstract

In this paper, we study how an oligopolist influences the coalition structure in federated cloud markets. Specifically, we use cooperative game theory to model the circumstances under which a cloud provider prefers to join a cloud federation vis-a-vis consider taking a price offer made by an oligopolist. We consider two price offering strategies for an oligopolist: non-adaptive and adaptive. In non-adaptive strategy, an oligopolist makes a price offer to all the cloud providers simultaneously. It can be noted that the oligopolist can buy-out all the cloud providers by making a price offer which is equal to a core allocation and the total price offer made by the oligopolist is equal to the value of the grand coalition. In adaptive strategy, the oligopolist approaches the cloud providers one after another in a sequential manner. We show that by using the adaptive strategy, the oligopolist can buy-out all the cloud providers at a total price offer which is less than that of the non-adaptive strategy

Abstract

Unlike CPUs and GPUs, it is possible to use custom fixed-point data types, specified as a tuple (α, β), on FPGAs. The parameters α and β denote the number of integral and fractional bitwidths respectively. The power and area savings while performing arithmetic operations on fixed-point data types are well known to be significant over using floating-point data types. In this paper, we propose a hybrid approach involving interval analysis and SMT solvers, for estimating integral bitwidths at different compute stages, in an image processing pipeline, specified using a domain-specific language (DSL) such as PolyMage. The DSL specification facilitates the compiler analysis to infer the underlying computational structure with ease. We also propose a simple and practical profile-driven greedy heuristic search technique for fractional bitwidth analysis. Using the Horn-Schunck Optical Flow benchmark program, we demonstrate where the conventional range analysis approaches fail, and how we overcome them using the hybrid technique proposed in this paper. The integral bitwidth estimates provided by the hybrid technique on the optical flow benchmark are up to 3x times better when compared with interval analysis.