Author details
Author's Published Works
Author details
Showing 1-20 of 20 results
- Sort
Filter Results
Show
Sort By
Results
EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture
Peiyan Dong;Jinming Zhuang;Zhuoping Yang;Shixin Ji;Yanyu Li;Dongkuan Xu;Heng Huang;Jingtong Hu;Alex K. Jones;Yiyu Shi;Yanzhi Wang;Peipei Zhou
While vision transformers (ViTs) have shown consistent progress in computer vision, deploying them for real-time decision-making scenarios (<1 ms) is challenging. Current computing platforms like CPUs, GPUs, or FPGA-based solutions struggle to meet this deterministic low-latency real-time requirement, even with quantized ViT models. Some approaches use pruning or sparsity to reduce the model size ...Show More
SDA: Low-Bit Stable Diffusion Acceleration on Edge FPGAs
Geng Yang;Yanyue Xie;Zhong Jia Xue;Sung-En Chang;Yanyu Li;Peiyan Dong;Jie Lei;Weiying Xie;Yanzhi Wang;Xue Lin;Zhenman Fang
2024 34th International Conference on Field-Programmable Logic and Applications (FPL)
Year: 2024 | Conference Paper |
This paper introduces SDA, the first effort to adapt the expensive stable diffusion (SD) model for edge FPGA deployment. First, we apply quantization-aware training to quantize its weights to 4 -bit and activations to 8 -bit (W 4 A 8) with a negligible accuracy loss. Based on that, we propose a high-performance hybrid systolic array (hybridSA) architecture that natively executes convolution and ...Show More
Research on Image Aesthetic Assessment based on Graph Convolutional Network
2024 IEEE International Conference on Multimedia and Expo (ICME)
Year: 2024 | Conference Paper |
Image aesthetic quality assessment (IAA) is aimed at predicting the general aesthetic evaluation of images by the public. However, the common neural network has a fixed size requirement for the input and ignores the close relation between image composition and aesthetic rating. Therefore, in this paper, we design and implement a concise and efficient IAA algorithm based on graph convolutional neur...Show More
TextCraftor: Your Text Encoder can be Image Quality Controller
Yanyu Li;Xian Liu;Anil Kag;Ju Hu;Yerlan Idelbayev;Dhritiman Sagar;Yanzhi Wang;Sergey Tulyakov;Jian Ren
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2024 | Conference Paper |
Diffusion-based text-to-image generative models, e.g., Stable Diffusion, have revolutionized the field of content generation, enabling significant advancements in areas like image editing and video synthesis. Despite their formidable capabilities, these models are not without their limitations. It is still challenging to synthesize an image that aligns well with the input text, and multiple runs w...Show More
DEFCON: Deformable Convolutions Leveraging Interval Search and GPU Texture Hardware
2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Year: 2024 | Conference Paper |
Deformable convolutions can improve detection accuracy in Convolution Neural Networks (CNNs) by leveraging flexible spatial sampling in augmenting kernels with learnable offsets. However, the resulting irregular memory access patterns and additional pixel lookup overhead introduced by deformable layers pose inherent challenges when executed on high-throughput devices such as GPUs. To address these...Show More
DACO: Pursuing Ultra-low Power Consumption via DNN-Adaptive CPU-GPU CO-optimization on Mobile Devices
Yushu Wu;Chao Wu;Geng Yuan;Yanyu Li;Weichao Guo;Jing Rao;Xipeng Shen;Bin Ren;Yanzhi Wang
2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Year: 2024 | Conference Paper |
As Deep Neural Networks (DNNs) become popular in mobile systems, their high computational and memory demands make them major power consumers, especially in limited-budget scenarios. In this paper, we propose DACO, a DNN-Adaptive CPU-GPU CO-optimization technique, to reduce the power consumption of DNNs. First, a resource-oriented classifier is proposed to quantify the computation/memory intensity ...Show More
Rethinking Vision Transformers for MobileNet Size and Speed
Yanyu Li;Ju Hu;Yang Wen;Georgios Evangelidis;Kamyar Salahi;Yanzhi Wang;Sergey Tulyakov;Jian Ren
2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Year: 2023 | Conference Paper |
Cited by: Papers (71)
With the success of Vision Transformers (ViTs) in computer vision tasks, recent arts try to optimize the performance and complexity of ViTs to enable efficient deployment on mobile devices. Multiple approaches are proposed to accelerate attention mechanism, improve inefficient designs, or incorporate mobile-friendly lightweight convolutions to form hybrid architectures. However, ViT and its varian...Show More
MOC: Multi-Objective Mobile CPU-GPU Co-Optimization for Power-Efficient DNN Inference
Yushu Wu;Yifan Gong;Zheng Zhan;Geng Yuan;Yanyu Li;Qi Wang;Chao Wu;Yanzhi Wang
2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD)
Year: 2023 | Conference Paper |
Cited by: Papers (1)
With the emergence of DNN applications on mobile devices, plenty of attention has been attracted to their optimization. However, the impact of DNN inference tasks on device power consumption is still a lack of comprehensive study. In this work, we propose MOC, a Multi-Objective deep reinforcement learning-assisted DNN inference stage-adaptive CPU-GPU Co-optimization approach. We find through exper...Show More
Fast and Fair Medical AI on the Edge Through Neural Architecture Search for Hybrid Vision Models
Changdi Yang;Yi Sheng;Peiyan Dong;Zhenglun Kong;Yanyu Li;Pinrui Yu;Lei Yang;Xue Lin;Yanzhi Wang
2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD)
Year: 2023 | Conference Paper |
As edge devices become readily available and indispensable, there is an urgent need for effective and efficient intelligent applications to be deployed widespread. However, fairness has always been an issue, especially in edge medical applications. Although many approaches have been proposed to mitigate the unfairness problem, their edge performance is not desirable. By examining the fairness perf...Show More
Late Breaking Results: Fast Fair Medical Applications? Hybrid Vision Models Achieve the Fairness on the Edge
Changdi Yang;Yi Sheng;Peiyan Dong;Zhenglun Kong;Yanyu Li;Pinrui Yu;Lei Yang;Xue Lin
2023 60th ACM/IEEE Design Automation Conference (DAC)
Year: 2023 | Conference Paper |
As edge devices become readily available and indispensable, there is an urgent need for effective and efficient intelligent applications to be deployed widespread. However, fairness has always been an issue, especially in edge medical applications. Compared to convolutional neuron networks (CNNs), Vision Transformer (ViT) has a better ability to extract global information, which will contribute to...Show More
Pruning Parameterization with Bi-level Optimization for Efficient Semantic Segmentation on the Edge
Changdi Yang;Pu Zhao;Yanyu Li;Wei Niu;Jiexiong Guan;Hao Tang;Minghai Qin;Bin Ren;Xue Lin;Yanzhi Wang
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2023 | Conference Paper |
Cited by: Papers (9)
With the ever-increasing popularity of edge devices, it is necessary to implement real-time segmentation on the edge for autonomous driving and many other applications. Vision Transformers (ViTs) have shown considerably stronger results for many vision tasks. However, ViTs with the fullattention mechanism usually consume a large number of computational resources, leading to difficulties for real- ...Show More
ESRU: Extremely Low-Bit and Hardware-Efficient Stochastic Rounding Unit Design for Low-Bit DNN Training
Sung-En Chang;Geng Yuan;Alec Lu;Mengshu Sun;Yanyu Li;Xiaolong Ma;Zhengang Li;Yanyue Xie;Minghai Qin;Xue Lin;Zhenman Fang;Yanzhi Wang
2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Year: 2023 | Conference Paper |
Cited by: Papers (3)
Stochastic rounding is crucial in the low-bit (e.g., 8-bit) training of deep neural networks (DNNs) to achieve high accuracy. One of the drawbacks of prior studies is that they require a large number of high-precision stochastic rounding units (SRUs) to guarantee low-bit DNN accuracy, which involves considerable hardware overhead. In this paper, we use extremely low-bit SRUs (ESRUs) to save a larg...Show More
AirNN: Over-the-Air Computation for Neural Networks via Reconfigurable Intelligent Surfaces
Sara Garcia Sanchez;Guillem Reus-Muns;Carlos Bocanegra;Yanyu Li;Ufuk Muncuk;Yousof Naderi;Yanzhi Wang;Stratis Ioannidis;Kaushik Roy Chowdhury
Over-the-air analog computation allows offloading computation to the wireless environment through carefully constructed transmitted signals. In this paper, we design and implement the first-of-its-kind convolution that uses over-the-air computation and demonstrate it for inference tasks in a convolutional neural network (CNN). We engineer the ambient wireless propagation environment through reconf...Show More
Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization
Zhengang Li;Mengshu Sun;Alec Lu;Haoyu Ma;Geng Yuan;Yanyue Xie;Hao Tang;Yanyu Li;Miriam Leeser;Zhangyang Wang;Xue Lin;Zhenman Fang
2022 32nd International Conference on Field-Programmable Logic and Applications (FPL)
Year: 2022 | Conference Paper |
Cited by: Papers (36)
Vision transformers (ViTs) are emerging with significantly improved accuracy in computer vision tasks. However, their complex architecture and enormous computation/storage demand impose urgent needs for new hardware accelerator design methodology. This work proposes an FPGA-aware automatic ViT acceleration framework based on the proposed mixed-scheme quantization. To the best of our knowledge, thi...Show More
Neural Network-Based OFDM Receiver for Resource Constrained IoT Devices
Nasim Soltani;Hai Cheng;Mauro Belgiovine;Yanyu Li;Haoqing Li;Bahar Azari;Salvatore D'Oro;Tales Imbiriba;Tommaso Melodia;Pau Closas;Yanzhi Wang;Deniz Erdogmus;Kaushik Chowdhury
IEEE Internet of Things Magazine
Cited by: Papers (12)
Orthogonal Frequency Division Multiplexing (OFDM)-based waveforms are used for communication links in many current and emerging Internet of Things (IoT) applications, including the latest WiFi standards. For such OFDM-based transceivers, many core physical layer functions related to channel estimation, demapping, and decoding are implemented for specific choices of channel types and modulation sch...Show More
Fault-Tolerant Deep Neural Networks for Processing-In-Memory based Autonomous Edge Systems
Siyue Wang;Geng Yuan;Xiaolong Ma;Yanyu Li;Xue Lin;Bhavya Kailkhura
2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Year: 2022 | Conference Paper |
Cited by: Papers (5)
In-memory deep neural network (DNN) accelerators will be the key for energy-efficient autonomous edge systems. The resistive random access memory (ReRAM) is a potential solution for the non-CMOS-based in-memory computing platform for energy-efficient autonomous edge systems, thanks to its promising characteristics, such as near-zero leakage-power and non-volatility. However, due to the hardware in...Show More
NN-key: A Neural Network-Based Secret Key for Demapping OFDM Symbols
Nasim Soltani;Yanyu Li;Deniz Erdogmus;Yanzhi Wang;Kaushik Chowdhury
2022 IEEE 19th Annual Consumer Communications & Networking Conference (CCNC)
Year: 2022 | Conference Paper |
Generating custom modulation patterns as well as dynamically varying the mapping of the constellation points to their corresponding bit representations are some existing methods for mitigating eavesdropping attacks. In such cases, the custom symbol to bit mapping needs to be conveyed to the receiver through a secure and reliable channel. Instead of sending the representations of the modified symbo...Show More
RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
Sung-En Chang;Yanyu Li;Mengshu Sun;Weiwen Jiang;Sijia Liu;Yanzhi Wang;Xue Lin
2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Year: 2021 | Conference Paper |
Cited by: Papers (3)
This work proposes a novel Deep Neural Network (DNN) quantization framework, namely RMSMP, with a Row-wise Mixed-Scheme and Multi-Precision approach. Specifically, this is the first effort to assign mixed quantization schemes and multiple precisions within layers – among rows of the DNN weight matrix, for simplified operations in hardware inference, while preserving accuracy. Furthermore, this pap...Show More
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
Zhengang Li;Geng Yuan;Wei Niu;Pu Zhao;Yanyu Li;Yuxuan Cai;Xuan Shen;Zheng Zhan;Zhenglun Kong;Qing Jin;Zhiyu Chen;Sijia Liu;Kaiyuan Yang;Bin Ren;Yanzhi Wang;Xue Lin
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2021 | Conference Paper |
Cited by: Papers (15)
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed. Prior methods towards this goal, including model compression and network architecture search (NAS), are largely performed independently, and do not fully consider compiler-level optimizations which is a must-do for mobile a...Show More
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
Zhengang Li;Geng Yuan;Wei Niu;Pu Zhao;Yanyu Li;Yuxuan Cai;Xuan Shen;Zheng Zhan;Zhenglun Kong;Qing Jin;Zhiyu Chen;Sijia Liu;Kaiyuan Yang;Bin Ren;Yanzhi Wang;Xue Lin
Year: 2021 | Conference Paper |
Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework
Sung-En Chang;Yanyu Li;Mengshu Sun;Runbin Shi;Hayden K.-H. So;Xuehai Qian;Yanzhi Wang;Xue Lin
2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)
Year: 2021 | Conference Paper |
Cited by: Papers (68)
Deep Neural Networks (DNNs) have achieved extraordinary performance in various application domains. To support diverse DNN models, efficient implementations of DNN inference on edge-computing platforms, e.g., ASICs, FPGAs, and embedded systems, are extensively investigated. Due to the huge model size and computation amount, model compression is a critical step to deploy DNN models on edge devices....Show More
IEEE Account
Purchase Details
Profile Information
Need Help?
- US & Canada: +1 800 678 4333
- Worldwide: +1 732 981 0060
- Contact & Support
- About IEEE Xplore
- Contact Us
- Help
- Accessibility
- Terms of Use
- Nondiscrimination Policy
- Sitemap
- Privacy & Opting Out of Cookies
A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.