KNN-augmented attention transformer for hybrid deep-learning-based meteorological forecasting

Yuhao Lu; Haoran Guo

doi:10.1117/12.3049508

8 November 2024 KNN-augmented attention transformer for hybrid deep-learning-based meteorological forecasting

Yuhao Lu, Haoran Guo

Proceedings Volume 13416, Fourth International Conference on Advanced Algorithms and Neural Networks (AANN 2024); 1341612 (2024) https://doi.org/10.1117/12.3049508
Event: 2024 4th International Conference on Advanced Algorithms and Neural Networks, 2024, Qingdao, China

Abstract

Recent advances in deep learning have significantly impacted weather forecasting, where accurate predictions are crucial for effective decision-making across various sectors. This paper presents a novel approach to weather forecasting by introducing an advanced transformer-based model, which harnesses the capabilities of the Focused Transformer (FOT) framework with significant adaptations for meteorological applications. Central to our method is the substitution of the exact k-nearest-neighbor (k-NN) lookup with an approximate k-NN method, enhancing the model’s efficiency and scalability. This modification enables the transformer to dynamically integrate expansive temporal and spatial data sequences more effectively, crucial for accurate weather predictions. We adopt the contrastive training technique to refine the context scaling, which is critical for processing the complex dynamics of weather patterns. Performance evaluations on several weather datasets demonstrate that our model achieves superior forecasting accuracy, indicated by improvements in root mean squared error and mean absolute error metrics, compared to existing state-of-the-art models. Our findings suggest that applying transformer models with approximate k-NN lookups offers a promising direction for developing more robust and efficient weather forecasting systems.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Yuhao Lu and Haoran Guo "KNN-augmented attention transformer for hybrid deep-learning-based meteorological forecasting", Proc. SPIE 13416, Fourth International Conference on Advanced Algorithms and Neural Networks (AANN 2024), 1341612 (8 November 2024); https://doi.org/10.1117/12.3049508

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data modeling

Transformers

Atmospheric modeling

Performance modeling

Meteorology

Education and training

Weather forecasting

Show All Keywords

Keywords/Phrases

Search In:

Publication Years