[Purpose/significance] This study aims to explore the effectiveness and feasibility of using ChatGPT for
evaluating the innovativeness of academic papers. If generative artificial intelligence, represented by ChatGPT, can be
employed for paper innovativeness assessment, it has the potential to significantly improve the efficiency of innovative?
ness evaluation. [Method/process] By simulating real-world demands and application scenarios for evaluating paper
innovativeness, we conducted over 24 rounds of innovativeness evaluation experiments across six research domains:
life sciences, cellular immunology, marketing, economics, artificial intelligence, and information resource management.
Content analysis was employed to thoroughly examine the text generated by ChatGPT during the process of evaluating
paper innovativeness, enabling us to study its approach, criteria, and feasibility. [Result/conclusion] The study re?
vealed that ChatGPT demonstrated excellent performance in the task of evaluating paper innovativeness, displaying
high comprehensiveness and scientific rigor in its evaluation methods and criteria. It effectively accomplished paper in?
novativeness evaluations across these research fields. At present, while ChatGPT's stability, accuracy, and authenticity
may have some minor limitations, it can still be considered an efficient auxiliary tool for evaluating paper innovative?
ness in relevant tasks. The identified shortcomings of using ChatGPT for paper innovativeness evaluation are likely to
be gradually addressed with the advancement of prompt engineering, opening up of Internet access, and increased avail?
ability of academic databases.
Wang Yaqi
,
Cao Shujin
. The Effect and Feasibility Analysis of ChatGPT Used in Paper Innovativeness Evaluation[J]. Information and Documentation Services, 2023
, 44(5)
: 28
-38
.
DOI: 10.12154/j.qbzlgz.2023.05.003