讲座信息
报 告 人:史成春 副教授(伦敦政经学院)
报告时间:8月24日周一15:30-17:00
报告地点: 阜成路综合楼1116会议室
Group relative policy optimization (GRPO), a core methodological component of DeepSeekMath and DeepSeek-R1, has emerged as a cornerstone for scaling reasoning capabilities of large language models. Despite its widespread adoption and the proliferation of follow-up works, the theoretical properties of GRPO remain less studied. This paper provides a unified framework to understand GRPO through the lens of classical U-statistics. We demonstrate that the GRPO policy gradient is inherently a U-statistic, allowing us to characterize its mean squared error(MSE), derive the finite-sample error bound and asymptotic distribution of the suboptimality gap for its learned policy. Our findings reveal that GRPO is asymptotically equivalent to an oracle policy gradient algorithm – one with access to a value function that quantifies the goodness of its learning policy at each training iteration – and achieves asymptotically optimal performance within a broad class of policy gradient algorithms. Furthermore, we establish a universal scaling law that offers principled guidance for selecting the optimal group size. Empirical experiments further validate our theoretical findings, demonstrating that the optimal group size is universal,and verify the oracle property of GRPO.
主讲人简介
Chengchun is an Associate Professor in the Department of Statistics at LSE. He works at the interface of RL,LLMs and statistics, with applications to ride-sharing and healthcare.His work brings to light the relevanceand significance of statistical leaming in AI, and demonstrates the usefulness of RL as a framework for policyevaluation and A/B testing in two-sided marketplaces. Chengchun has published over 70 papers, with majorityof them accepted in prestigious statistical joumals (JRSSB, JASA, AoS) and top machine learning venues(NeurlPS, ICML, CVPR, ICLR, KDD).His outstanding contributions have been recognized with esteemedawards such as the Peter Gavin Hall IMS Early Career Prize, IMS Tweedie Award and the Royal StatisticalSociety Research Prize. He has served as the area editor of NeurlPS and associate editors of prestigious joumals JRSS-B, JASA and AoAS.