Chinese abstractive summarization based on pre-trained module assisted with noise co-attention

Xiaojun Bi; Sa Yang; Weizheng Qiao

doi:10.1117/12.3005079

9 October 2023 Chinese abstractive summarization based on pre-trained module assisted with noise co-attention

Xiaojun Bi, Sa Yang, Weizheng Qiao

Proceedings Volume 12791, Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023); 127911W (2023) https://doi.org/10.1117/12.3005079
Event: Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023), 2023, Qingdao, SD, China

Abstract

The encoder-decoder framework underpins the majority of Chinese abstract generative models. Although the generated summaries are comparable to the source text, they invariably suffer from generating inaccurate or redundant words, which can result in imprecise content or insufficient summaries. To address this issue, we propose a Roberta-based model with noise co-attention. In our way, we propose a novel noise co-attention to correct and remove the learned biased and redundant information from the source text. We first use the pre-trained model ROBERTA to obtain high-quality word vector representations and then build UNILM based on the first step, in particular adding noise co-attention layer to guide the summary generation process. Related experiments are being carried out on the large Chinese abstract dataset LCSTS, using ROUGE as the evaluation metric. According to the results, our invention surpasses numerous cutting-edge models for the summary generation and offers greater improvements over baseline methods.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Xiaojun Bi, Sa Yang, and Weizheng Qiao "Chinese abstractive summarization based on pre-trained module assisted with noise co-attention", Proc. SPIE 12791, Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023), 127911W (9 October 2023); https://doi.org/10.1117/12.3005079

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available