Paper
3 February 2023 A web log real-time analysis platform based on stream computing
Jun Zhang, Li Zhang, Peiyi Tang, Taizhi Lv
Author Affiliations +
Proceedings Volume 12511, Third International Conference on Computer Vision and Data Mining (ICCVDM 2022); 125111W (2023) https://doi.org/10.1117/12.2660112
Event: Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 2022, Hulun Buir, China
Abstract
With the rapid development of Internet technology, web applications have been widely used. How to ensure the reliability and high performance of web applications has become the focus of web site management. When a web application provides services, huge web logs are generated. These logs contain a great deal of information about users access to this web application. Real-time analysis of web logs can obtain system performance indicators and bottlenecks. In order to improve the reliability and performance of web applications, a real-time web log analysis platform based on stream computing is designed and implemented. The platform collects web log data by Flume, realizes data flow by Kafka message queue, analyzes web log by the Flink stream computing platform, stores the computing results in Redis for real-time query, and in Doris for historical data query. The availability of the platform is proved by running in a real environment, and it can improve the performance and reliability of web applications.
© (2023) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Jun Zhang, Li Zhang, Peiyi Tang, and Taizhi Lv "A web log real-time analysis platform based on stream computing", Proc. SPIE 12511, Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 125111W (3 February 2023); https://doi.org/10.1117/12.2660112
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Data acquisition

Data analysis

Statistical analysis

Data conversion

Data processing

Visualization

Data visualization

Back to Top