CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 12174 > Article
Paper
22 April 2022 An improved algorithm of EfficientNet with Self Attention mechanism
Yuxiang Wang
Author Affiliations +
Proceedings Volume 12174, International Conference on Internet of Things and Machine Learning (IoTML 2021); 121740W (2022) https://doi.org/10.1117/12.2628629
Event: International Conference on Internet of Things and Machine Learning (IoTML 2021), 2021, Shanghai, China
Abstract
Traditional convolutional neural networks usually consider both resource budget and operation cost. However, if there is a better operating environment and more advanced resources available, a more accurate and faster algorithm to solve some problems of existing networks can be provided. In this paper, the author thoroughly studied the Self Attention mechanism and EfficientNet network that Google has released and combined the advantages of the two networks. Based on this result, the author proposes an improved algorithm of EfficientNet with Self Attention mechanism. Before introducing EfficientNet pre-training model to start training, EfficientNet uses simple but efficient composite coefficients to uniformly scale all dimensions of depth, width and resolution. A simple Self Attention network structure is trained in advance to process data images. This will enable the pre-training network to focus as much as possible on the data and image features with a large amount of information and large gap before entering EfficientNet training. After that, the pre-training network will train the model. Through experiments, it is found that the model has higher quality. The improved algorithm showed extremely high accuracy and fast convergence speed on CIFAR-100, OxFlowers and ImageNet, and three other transfer learning datasets. The accuracy of the improved algorithm network with efficientnet-b4 as the pre-training model on the three data sets is greater than 95%. Compared with the original EfficientNet, the comprehensive improvement is more than 10%, and the running speed is increased by about 30%. Compared with the classic convnet, it can be increased by 20% and the running speed is increased by more than 50%.
© (2022) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Yuxiang Wang "An improved algorithm of EfficientNet with Self Attention mechanism", Proc. SPIE 12174, International Conference on Internet of Things and Machine Learning (IoTML 2021), 121740W (22 April 2022); https://doi.org/10.1117/12.2628629
ACCESS THE FULL ARTICLE
ORGANIZATIONAL
Sign in with credentials provided by your organization.
INSTITUTIONAL
Select your institution to access the SPIE Digital Library.
SELECT YOUR INSTITUTION
PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
PERSONAL SIGN IN
No SPIE Account? Create one
PURCHASE THIS CONTENT
SUBSCRIBE TO DIGITAL LIBRARY
50 downloads per 1-year subscription
Members: $195
Non-members: $335 ADD TO CART
25 downloads per 1 - year subscription
Members: $145
Non-members: $250 ADD TO CART
PURCHASE SINGLE ARTICLE
Includes PDF, HTML & Video, when available
Members: $17.00
Non-members: $21.00 ADD TO CART
PROCEEDINGS
 9 PAGES
DOWNLOAD PAPER SAVE TO MY LIBRARY
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Data modeling
Convolution
Evolutionary algorithms
Detection and tracking algorithms
Feature extraction
Image processing
Neural networks
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top