Ganqu Cui Ph.D Shanghai AI Laboratory

Biography

I am a researcher in Shanghai AI Laboratory. I obtained my Ph.D degree from THUNLP Lab, Dept. of Computer Science and Technology, Tsinghua University, advised by Prof. Zhiyuan Liu. Before that, I obtained my B.S. degree of Mathematics and Physics, Tsinghua University in 2019.

My research interests lie at LLM alignment and reinforcement learning. Previously, I did some research on representation learning on graphs, especially graph neural networks and their application.

We are hiring! If you are interested in building large reasoning models with reinforcement learning, please drop me an email (thu.cgq AT foxmail.com).

News

Publications [Google Scholar]

* indicates equal contribution. † indicates corresponding author.

• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
  Ganqu Cui*, Yuchen Zhang*, Jiacheng Chen*, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, Ning Ding.
  Preprint
  [code]
  
• TTRL: Test-time reinforcement learning
  Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biqing Qi, Youbang Sun, Ganqu Cui, Ning Ding, Bowen Zhou.
  Preprint
  [code]
  
• UltraIF: Advancing Instruction Following from the Wild
  Kaikai An*, Li Sheng*, Ganqu Cui†, Shuzheng Si, Ning Ding, Yu Cheng, Baobao Chang†.
  Preprint
  [code]
  
• Process Reinforcement through Implicit Rewards
  Ganqu Cui*, Lifan Yuan*, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, Jiarui Yuan, Huayu Chen, Kaiyan Zhang, Xingtai Lv, Shuo Wang, Yuan Yao, Xu Han, Hao Peng, Yu Cheng, Zhiyuan Liu, Maosong Sun, Bowen Zhou, Ning Ding.
  Preprint
  [code]
  
• Free Process Rewards without Process Labels
  Lifan Yuan*, Wendi Li*, Huayu Chen, Ganqu Cui†, Ning Ding†, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu†, Hao Peng.
  ICML 2025
  [code]
  
• Advancing LLM Reasoning Generalists with Preference Trees
  Lifan Yuan*, Ganqu Cui*†, Hanbin Wang*, Ning Ding†, Xingyao Wang, Jia Deng, Boji Shan, Huimin Chen, Ruobing Xie, Yankai Lin, Zhenghao Liu, Bowen Zhou, Hao Peng, Zhiyuan Liu†, Maosong Sun.
  ICLR 2025
  [code]
  
• Minicpm: Unveiling the potential of small language models with scalable training strategies (Oral)
  Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, Maosong Sun.
  COLM 2024
  [code]
  
• Controllable preference optimization: Toward controllable multi-objective alignment
  Yiju Guo*, Ganqu Cui*, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun.
  EMNLP 2024
  [code]
  
• UltraFeedback: Boosting Language Models with Scaled AI Feedback
  Ganqu Cui*, Lifan Yuan*, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, Maosong Sun.
  ICML 2024
  [code]
  
• Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
  Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, Maosong Sun.
  NeurIPS Datasets & Benchmarks 2023
  
• Removing Backdoors in Pre-trained Models by Regularized Continual Pre-training
  Biru Zhu*, Ganqu Cui*, Yangyi Chen, Yujia Qin, Lifan Yuan, Chong Fu, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu.
  TACL
  
• Tool Learning with Foundation Models
  Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu, Zhenning Dai, Lan Yan, Xin Cong, Yaxi Lu, Weilin Zhao, Yuxiang Huang, Junxi Yan, Xu Han, Xian Sun, Dahai Li, Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji, Zhiyuan Liu, Maosong Sun.
  ACM Computing Survey
  [code]
  
• Decoder Tuning: Efficient Language Understanding as Decoding
  Ganqu Cui, Wentao Li, Ning Ding, Longtao Huang, Zhiyuan Liu, Maosong Sun.
  ACL 2023
  [code]
  
• A Close Look into the Calibration of Pre-trained Language Models
  Yangyi Chen*, Lifan Yuan*, Ganqu Cui, Zhiyuan Liu, Heng Ji.
  ACL 2023
  [code]
  
• From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework
  Yangyi Chen*, Hongcheng Gao*, Ganqu Cui*, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji.
  Findings of ACL 2023
  
• Few-shot Classification with Hypersphere Modeling of Prototypes
  Ning Ding, Yulin Chen, Ganqu Cui, Xiaobin Wang, Hai-Tao Zheng, Zhiyuan Liu, Pengjun Xie
  Findings of ACL 2023
  
• Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP
  Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, Maosong Sun.
  EMNLP 2022
  [code]
  
• A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks (Spotlight)
  Ganqu Cui*, Lifan Yuan*, Bingxiang, He, Yangyi Chen, Zhiyuan Liu, Maosong Sun.
  NeurIPS Datasets & Benchmarks 2022
  [code]
  
• Moderate-fitting as a Natural Backdoor Defender for Pre-trained Language Models
  Biru Zhu, Yujia Qin, Ganqu Cui, Yangyi Chen, Weilin Zhao, Chong Fu, Yangdong Deng, Zhiyuan Liu, Jingang Wang, Wei Wu, Maosong Sun, Ming Gu.
  NeurIPS 2022
  [code]
  
• Exploring the Universal Vulnerability of Prompt-based Learning Paradigm
  Lei Xu, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Zhiyuan Liu
  Findings of NAACL 2022
  [code]
  
• Prototypical Verbalizer for Prompt-based Few-shot Tuning
  Ganqu Cui, Shengding Hu, Ning Ding, Longtao Huang, Zhiyuan Liu.
  ACL 2022
  [code]
  
• Evaluating Modules in Graph Contrastive Learning
  Ganqu Cui, Yufeng Du, Cheng Yang, Jie Zhou, Liang Xu, Lifeng Wang, Zhiyuan Liu.
  Preprint
  
• Graph Neural Networks: A Review of Methods and Applications
  Jie Zhou*, Ganqu Cui*, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun.
  AI Open 2021
  
• Adaptive Graph Encoder for Attributed Graph Embedding
  Ganqu Cui, Jie Zhou, Cheng Yang, Zhiyuan Liu.
  KDD 2020
  [code]
  
• Multi-scale Information Diffusion Prediction with Reinforced Recurrent Networks
  Cheng Yang, Jian Tang, Maosong Sun, Ganqu Cui, Zhiyuan Liu.
  IJCAI 2019
  
• Machine-Learning-Driven Matrix Ordering for Power Grid Analysis
  Ganqu Cui, Wenjian Yu, Xin Li, Zhiyu Zeng, Ben Gu.
  DATE 2019
  

Honors & Awards

WAIC Yunfan Rising Star Award, 2025

Tsinghua Outstanding Doctoral Dissertation (Top 10%)

Tsinghua Outstanding Graduate (Top 4%)

Longfor Scholarship, 2022, 2023

Tsinghua-Sohu R&D Scholarship, 2022

Tsinghua-Tang Junyuan Scholarship, 2023

Invited Talks

2025.05, The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models @ Tsinghua Foundation Model Center

2025.02, Process Reinforcement through Implicit Rewards @ Apple, NVIDIA, Tencent, Ant Group

2025.01, Process Reinforcement through Implicit Rewards @ Huawei Noah's Ark Lab

Projects

• OpenBackdoor
  An open-scource toolkit for textual backdoor attack and defense
  
• GNN Papers
  10k star GNN paper reading list
  
• OpenNE-pytorch
  An open source toolkit for Network Embedding
  

Professional Activities

Conference Reviews:
The Web Conference 2021
EMNLP 2022
NeurIPS 2022
ACL 2023
SIGIR 2023

Journal Reviews:
IEEE Transactions on Knowledge and Data Engineering (TKDE)
AI Open

Teaching Assistant

2019-2023	Spring	TA in Natural Language Processing
2022-2023	Fall	TA in Writing and Communication