Mr. Jing-de Bu Profile

Jing-de Bu

SPIE Involvement:

Author

Publications (4)

Proceedings Article | 9 October 2023 Paper

Research on optimization of convolution back propagation on domestic heterogeneous platforms

Zhan Yang, Fushuai Li, Yongqing Chen, Jingde Bu, Rong Dai

Proceedings Volume 12791, 127910X (2023) https://doi.org/10.1117/12.3005093

KEYWORDS: Convolution, Mathematical optimization, Matrices, Neural networks, Algorithm development, Parallel computing, Education and training, Computer programming, Computer hardware, Computer architecture

Read Abstract +

Proceedings Article | 21 July 2023 Paper

Optimization of convolutional backpropagation operators based on domestic accelerators

Fushuai Li, Zhan Yang, Yongqing Chen, Jingde Bu, Jinliang Jiang

Proceedings Volume 12717, 127171X (2023) https://doi.org/10.1117/12.2684767

KEYWORDS: Convolution, Mathematical optimization, Matrices, Convolutional neural networks, Deep learning, Machine learning, Neural networks

Read Abstract +

Proceedings Article | 8 May 2023 Paper

Optimizing Winograd convolution on GPUs via multithreaded communication

Yunqing An, Bin Li, Jingde Bu, Ya Gao

Proceedings Volume 12635, 126350W (2023) https://doi.org/10.1117/12.2679935

KEYWORDS: Convolution, Mathematical optimization, Tunable filters, Evolutionary algorithms, Data processing, Matrix multiplication, Convolutional neural networks, Deep learning, Data communications, Computer programming

Read Abstract +

Proceedings Article | 13 April 2023 Paper

Research on large-scale training optimization for transformer of domestic accelerator

Yan Zhu, Chen Hu, Shan-shan Ai, Jing-de Bu, Meng-zhi Han, Lin Han

Proceedings Volume 12605, 126050J (2023) https://doi.org/10.1117/12.2673331

KEYWORDS: Education and training, Transformers, Mathematical optimization, Matrix multiplication, Computer hardware, Data communications, Matrices, Performance modeling, Data transmission, Network architectures

Read Abstract +

Large-scale network models based on transformer architecture have strong versatility in many fields. Due to the computation-intensive and large-scale characteristics of the model, large-scale training on domestic heterogeneous accelerators is restricted by aspects such as computing and communication efficiency, resulting in poor training performance. Aiming at this problem, the hot functions and performance bottlenecks in the training process are studied and analyzed, and the corresponding performance optimization methods are proposed based on the hardware characteristics of domestic heterogeneous accelerators. In order to solve the problem of low performance in low accuracy training, the low accuracy package optimization is carried out for the underlying matrix multiplication core operator. To solve the problem of significant startup delay of kernel function caused by fine-grained core operators, the LightSeq framework is transplanted on the domestic heterogeneous platform for the first time, and the core fine-grained operators are specially optimized to adapt to the hardware structure according to the characteristics of network structure to accelerate the training process. In large-scale training, in order to solve the problem of low bandwidth in cross node communication, distributed communication optimization is carried out from two levels of data transmission and hardware topology, and communication efficiency is improved by reducing the frequency of communication and increasing the communication bandwidth. The experimental results show that using the WMT '14 English-German translation dataset, the performance is improved by two times after optimization on a single node without loss of training accuracy. The computing scale is gradually expanded to 128 nodes (512 accelerator cards) for large-scale distributed training and verification. Under the premise of ensuring performance improvement, the scalability can reach more than 90% in 256 accelerator cards.

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Keywords/Phrases

Search In:

Publication Years