Test Analysis of Typical Tasks in the Information Field Performed by Generative AI Large Models at Home and Abroad

  • Zhao Bang ,
  • Cao Shujin
Expand
  • 1 School of Information Management, Sun Yat-sen University, Guangzhou, 510006; 2 Information Management Research Institute,Shandong University of Technology, Zibo, 255000)

Online published: 2023-09-21

Abstract

[Purpose/significance] In the face of the unprecedented changes brought about by the advancement of AI and the development of large language models like ChatGPT in the field of information science, the effectiveness and re? liability of generative AI large language models in performing typical tasks in the field of information science are ana? lyzed in many ways, in order to provide advice for efficiently utilizing their abilities to assist scientific research and practical work. [Method/process] Questions were formed by combining the research and practical work content of vari? ous topics in the information field. Tests, comparisons and prompt engineering explorations were carried out on Gpt3.5-Turbo and ChatGLM-6B respectively, and experts-based rating method is used to score, then analyze and evalu? ate the nine abilities of large language models in detail. [Result/conclusion] The large language model has shown strong ability in performing most typical information tasks. Using the prompt engineering can greatly improve the supe? riority of the large language model. Combining external tools can effectively make up for the shortcoming of the large language model itself. Fully exploiting the potential of a large language model will bring about a huge improvement in information work efficiency.

Cite this article

Zhao Bang , Cao Shujin . Test Analysis of Typical Tasks in the Information Field Performed by Generative AI Large Models at Home and Abroad[J]. Information and Documentation Services, 2023 , 44(5) : 6 -17 . DOI: 10.12154/j.qbzlgz.2023.05.001

Outlines

/