Exploring ChatGPT's code summarization capabilities: an empirical study

Kang Wang; Guohua Shen; Zhiqiu Huang; Xinbo Zhang

doi:10.1117/12.3051717

18 November 2024 Exploring ChatGPT's code summarization capabilities: an empirical study

Kang Wang, Guohua Shen, Zhiqiu Huang, Xinbo Zhang

Proceedings Volume 13403, International Conference on Algorithms, High Performance Computing, and Artificial Intelligence (AHPCAI 2024) ; 134032I (2024) https://doi.org/10.1117/12.3051717
Event: International Conference on Algorithms, High Performance Computing, and Artificial Intelligence, 2024, Zhengzhou, China

Abstract

Various automated code summarization techniques enhance the efficiency and accuracy of code annotations, enabling succinct natural language comments for code snippets. Recently, large language models (LLMs) have significantly improved natural language processing tasks. Among them, ChatGPT, based on the GPT-3.5 architecture, has gained widespread attention in academia and industry. Previous studies have also tested the ability of ChatGPT in code summarization, designing heuristic questions to explore an appropriate prompt that can guide ChatGPT to generate comments. In contrast, we have designed a more targeted and adaptive suggestion word strategy to study the impact of prompt design on model generation summary. Additionally, we have made extensive data fine-tuning to enhance ChatGPT's ability in code summarization tasks. The experimental results demonstrate that our prompt strategy has significantly improved the quality of code summaries generated by ChatGPT compared to previous studies, but still falls short of the SOTA model.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Kang Wang, Guohua Shen, Zhiqiu Huang, and Xinbo Zhang "Exploring ChatGPT's code summarization capabilities: an empirical study", Proc. SPIE 13403, International Conference on Algorithms, High Performance Computing, and Artificial Intelligence (AHPCAI 2024) , 134032I (18 November 2024); https://doi.org/10.1117/12.3051717

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Education and training

Data modeling

Performance modeling

Design

Computer programming languages

Data analysis

Deep learning

Show All Keywords

Keywords/Phrases

Search In:

Publication Years