A web crawler-based method for collecting information on investment promotion enterprises

Jingyao Sun; Shengnan Zhang; mengli Dai

doi:10.1117/12.2683931

14 June 2023 A web crawler-based method for collecting information on investment promotion enterprises

Jingyao Sun, Shengnan Zhang, mengli Dai

Proceedings Volume 12708, 3rd International Conference on Internet of Things and Smart City (IoTSC 2023）; 127081U (2023) https://doi.org/10.1117/12.2683931
Event: 3rd International Conference on Internet of Things and Smart City (IoTSC 2023）, 2023, Chongqing, China

Abstract

Data acquisition is a prerequisite for performing big data analytics. However, as the diversity and timeliness of data increase, the complexity of data collection also increases. In this paper, we take enterprise data on a big data investment platform as the research object, and design two data collection models, static data collection based on incremental crawlers and dynamic data collection based on query topic crawlers, for the static and dynamic characteristics of this data. In the experiments, this paper tests the effectiveness of these two web crawler methods and proves that they can collect static and dynamic investment data comprehensively and accurately. Thus, this study provides an effective data collection scheme that helps improve the accuracy and reliability of big data analysis.

Citation Download Citation

Jingyao Sun, Shengnan Zhang, and mengli Dai "A web crawler-based method for collecting information on investment promotion enterprises", Proc. SPIE 12708, 3rd International Conference on Internet of Things and Smart City (IoTSC 2023）, 127081U (14 June 2023); https://doi.org/10.1117/12.2683931

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available