SuperCLUE: GPT4 continues to rank first in the Chinese large model list in October 2023

Recently, SuperCLUE released the latest Chinese large model list for October . GPT4 continued to dominate the list and ranked first. Vivo's self-developed large model vivoLM ranked fourth with a score of 70.74, ranking first among domestic large models .

Following vivoLM are Moonshot from Dark Side of the Moon, Wenxin Yiyan 4.0 from Baidu and SenseChat 3.0 from SenseTime.

SuperCLUE mainly examines the performance of large models in Chinese language capabilities, including hundreds of tasks in four major capability dimensions: professional knowledge and skills, language understanding and generation, AI agents, and security .

This evaluation selected 20 of the most representative general-purpose large language models at home and abroad. Compared with September, Moonshot from Dark Side of the Moon, Wenxin Yiyan 4.0 from Baidu, Spark V3.0 from iFlytek, vivoLM from vivo and Qwen-14B from Alibaba Cloud were added.

The evaluation data set for this test consists of 3,754 new test questions, including 606 multi-round short-answer questions and 3,148 objective multiple-choice questions. Finally, five major rankings including the overall ranking were selected.

The evaluation results show that the domestic first-tier large model structure has basically been formed. The top few Chinese large models are already very close to GPT3.5, but are still far away from GPT4. There is no sign of benchmarking or rivaling GPT4 .

SuperCLUE also believes that a general large model that will surpass GPT3.5 in all aspects will appear in the fourth quarter of this year , but how to surpass GPT4 will become a new challenge facing all Chinese model research and development institutions.

Zikuai Technology