A study on extraction QA-model based on disturbing word embedding and adversarial self-attention mechanism

Zhuopei Yu

doi:10.1117/12.3034371

16 October 2024 A study on extraction QA-model based on disturbing word embedding and adversarial self-attention mechanism

Zhuopei Yu

Author Affiliations +

Proceedings Volume 13291, Ninth International Symposium on Advances in Electrical, Electronics, and Computer Engineering (ISAEECE 2024); 132914Q (2024) https://doi.org/10.1117/12.3034371
Event: Ninth International Symposium on Advances in Electrical, Electronics, and Computer Engineering (ISAEECE 2024), 2024, Changchun, China

Abstract

Question-and-answer model research has recently been continuously explored in the NLP field, such as the development of human-machine interaction. But the extent to which these models truly understand language remains questionable, and in the face of antagonistic examples, relying too much on some of the word characteristics, makes the model unable to accomplish the tasks that require language understanding and reasoning ability, and the deficiencies of the model are magnified. Therefore, this paper proposes a method (PEC) based on disturbing word embedding and antagonizing self-attention mechanism, which is to obtain semantically retaining adversarial samples by disturbing verb embedding and reduce the noise effect of word embedding process on model training. Through contextual global constraint countermeasure training to enhance the model's deep mining of semantic information, which adds anti-self-attention mechanism to reduce the model's dependence on false features to improve the model's prediction of answers using simple matching strategies. The effectiveness and superiority of PEC method in enhancing the accuracy and robustness of question and answer model are verified by numerous experiments on multiple datasets.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Zhuopei Yu "A study on extraction QA-model based on disturbing word embedding and adversarial self-attention mechanism", Proc. SPIE 13291, Ninth International Symposium on Advances in Electrical, Electronics, and Computer Engineering (ISAEECE 2024), 132914Q (16 October 2024); https://doi.org/10.1117/12.3034371

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE