Zheng Zhang
Associate Professor
Research Interests
Make your next move.
Prepare your application
Review drafts before you send them.
Explore related searches
About
Zheng Zhang is an Assistant Professor at the University of California Santa Barbara, USA. His research interests encompass Bayesian tensorized neural networks, tensor decomposition, and uncertainty quantification, particularly in the context of electronic and photonic integrated circuits. Recent articles by Professor Zhang highlight advancements in efficient data processing and optimization methods for high-dimensional datasets.
Articles (10)
Back propagation (BP) is the default solution for gradient computation in neural network training. However, implementing BP-based training on various edge devices such as FPGA, microcontrollers (MCUs), and analog computing platforms faces multiple major challenges, such as the lack of hardware resources, long time-to-market, and dramatic errors in a low-precision setting. This article presents a simple BP-free training scheme on an MCU, which makes edge training hardware design as easy as inference hardware design. We adopt a quantized zeroth-order method to estimate the gradients of quantized model parameters, which can overcome the error of a straight-through estimator in a low-precision BP scheme. We further employ a few dimension reduction methods (e.g., node perturbation, sparse training) to improve the convergence of zeroth-order training. Experiment results show that our BP-free training achieves comparable performance as BP-based training on adapting a pre-trained image classifier to various corrupted data on resource-constrained edge devices (e.g., an MCU with 1024-KB SRAM for dense full-model training, or an MCU with 256-KB SRAM for sparse training). This method is most suitable for application scenarios where memory cost and time-to-market are the major concerns, but longer latency can be tolerated.
Year:
2025
Year:
2021
Year:
2021
Year:
2021
Year:
2021
Year:
2021
Year:
2021
Year:
2020
Year:
2020
Year:
2020
Make your next move.
Prepare your application
Review drafts before you send them.