High availability and fault tolerance design of big data analysis platform

Jianmin Xie; Zhaolong Teng

doi:10.1117/12.3032801

12 July 2024 High availability and fault tolerance design of big data analysis platform

Jianmin Xie, Zhaolong Teng

Proceedings Volume 13185, International Conference on Communication, Information, and Digital Technologies (CIDT2024) ; 1318505 (2024) https://doi.org/10.1117/12.3032801
Event: International Conference on Communication, Information and Digital Technologies, 2024, Wuhan, China

Abstract

In response to the fact that big data analysis platforms often only focus on certain specific aspects of high availability and fault tolerance, and their availability and fault tolerance are relatively low, this article takes power data as the object, combines cloud computing to build a big data analysis platform, and conducts research on the optimization design of platform availability and fault tolerance. Firstly, relevant experimental data such as equipment status data and power supply network data of a certain power company can be collected. Then, in order to meet the needs of high availability and fault tolerance of the platform, cloud computing multi area deployment and elastic resource management can be adopted, and load balancing technology can be used to evenly allocate requests to multiple nodes. By utilizing fault-tolerant routing mechanisms, faulty nodes can be detected and bypassed in a timely manner. Finally, a power big data analysis platform that integrates cloud computing can be constructed, and the high availability and fault tolerance of the experimental platform can be verified. The experimental results show that the platform for cloud computing based big data analysis optimization only takes 0.68 seconds to recover from failures, which is 1.27 seconds less than traditional big data analysis platforms. The platform data integrity is as high as 98.60%, indicating that the experimental platform has high availability and fault tolerance.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Jianmin Xie and Zhaolong Teng "High availability and fault tolerance design of big data analysis platform", Proc. SPIE 13185, International Conference on Communication, Information, and Digital Technologies (CIDT2024) , 1318505 (12 July 2024); https://doi.org/10.1117/12.3032801

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available