I am a third-year master student from the Department of Computer Science and Technology, Tsinghua University. I am fortunately supervised by Professor Zhiyuan Liu.

My current research focuses on the scalable oversight and comprehensive evaluation of large models, aiming to uncover predictable scaling laws and unlock the potential of model training.

News!

🔥 2024.4 We release UltraEval paper.A user-friendly LLM evaluation framework characterized by lightweight, comprehensiveness, modularity, and efficiency!

🔥 2024.2 We release OlympiadBench. An Olympiad-level bilingual multimodal scientific benchmark!

🔥 2024.2 MiniCPM has been released, using UltraEval as its evaluation framework!

🔥 2023.11 We release UltraEval. A platform for easy evaluation of LLMs’ well-rounded ability!

🔥 2023.9 The first sucessful attempt on predictable scaling was released!

Publications

• MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies. Preprint

  Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, Maosong Sun

• UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs. Preprint

  A user-friendly LLM evaluation framework characterized by lightweight, comprehensiveness, modularity, and efficiency!

  Chaoqun He, Renjie Luo, Shengding Hu, Yuanqian Zhao, Jie Zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

• OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems. Preprint-ACL 2024

  An Olympiad-level bilingual multimodal scientific benchmark

  Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

• Predicting Emergent Abilities with Infinite Resolution Evaluation. Preprint-ICLR 2024

  The first work that achieves predictable scaling besides GPT-4

  Shengding Hu, Xin Liu, Xu Han, Xinrong Zhang, Chaoqun He, Weilin Zhao, Yankai Lin, Ning Ding, Zebin Ou, Guoyang Zeng, Zhiyuan Liu, Maosong Sun

Selected Projects

Awards

• Friends of Tsinghua-Wuhan Excellence Scholarship 2023.

• The CCF Elite Collegiate Award 2019.

  Only 73 in the whole country.

• Global Second Prize in the 2019 ASC World Student Supercomputer Competition.

• National Encouragement scholarship 2017.