## A Survey of Reinforcement Learning for Large Reasoning Models
[](https://github.com/sindresorhus/awesome) [](https://arxiv.org/abs/2509.08827) [](https://github.com/TsinghuaC3I/Awesome-RL-Reasoning-Recipes) [](https://huggingface.co/papers/2509.08827) [](https://x.com/OkhayIea/status/1965989894163235111)
> We welcome everyone to open an issue for any related work we havenβt discussed, and weβll try to address it in the next release!
## π News
- **[2025-11-05]** π₯ Excited to release our paper list about **Memory for Agents**, covering breakthroughs in Context Management and Learning from Experience powering self-improving AI agents. Check it out: [GitHub](https://github.com/TsinghuaC3I/Awesome-Memory-for-Agents)
- **[2025-10]** π Honored to give talks at [BAAI](https://event.baai.ac.cn/activities/961), [Qingke Talk](https://qingkeai.online/archives/0h3Cm8Bi) and Tencent Wiztalk! Here are the [slides](Survey@RL4LRM-v1.pdf).
- **[2025-09-18]** π We update the full list of papers in the category structure of the survey!
- **[2025-09-12]** π Our survey was ranked **#1 Paper of the Day** on π€ [Hugging Face Daily Papers](https://huggingface.co/papers/2509.08827)!
- **[2025-09-11]** π₯ Excited to release our **RL for LRMs Survey**! Weβll be updating the full list of papers in with a new category structure soon. Check it out: [Paper](https://huggingface.co/papers/2509.08827).
- **[2025-08-15]** π₯ Introducing **SSRL**: an investigation for Agentic Search RL without reliance on external search engine. Check it out: [GitHub](https://github.com/TsinghuaC3I/SSRL) and [Paper](https://arxiv.org/abs/2508.10874).
- **[2025-05-27]** π₯ Introducing **MARTI**: A Framework for LLM-based Multi-Agent Reinforced Training and Inference. Check it out: [Github](https://github.com/TsinghuaC3I/MARTI).
- **[2025-04-23]** π₯ Introducing **TTRL**: an open-source solution for online RL on data without ground-truth labels, especially test data. Check it out: [Github](https://github.com/PRIME-RL/TTRL) and [Paper](https://arxiv.org/abs/2504.16084).
- **[2025-03-20]** π₯ We are excited to introduce collection of papers and projects on RL for reasoning models!
## π Citation
If you find this survey helpful, please cite our work:
```bibtex
@article{zhang2025survey,
title={A survey of reinforcement learning for large reasoning models},
author={Zhang, Kaiyan and Zuo, Yuxin and He, Bingxiang and Sun, Youbang and Liu, Runze and Jiang, Che and Fan, Yuchen and Tian, Kai and Jia, Guoli and Li, Pengfei and others},
journal={arXiv preprint arXiv:2509.08827},
year={2025}
}
```
## π Contents
- [A Survey of Reinforcement Learning for Large Reasoning Models](#a-survey-of-reinforcement-learning-for-large-reasoning-models)
- [π News](#-news)
- [π Citation](#-citation)
- [π Contents](#-contents)
- [πΊοΈ Overview](#οΈ-overview)
- [π Paper List](#-paper-list)
- [Frontier Models](#frontier-models)
- [Reward Design](#reward-design)
- [Generative Rewards](#generative-rewards)
- [Dense Rewards](#dense-rewards)
- [Unsupervised Rewards](#unsupervised-rewards)
- [Rewards Shaping](#rewards-shaping)
- [Policy Optimization](#policy-optimization)
- [Policy Gradient Objective](#policy-gradient-objective)
- [Critic-based Algorithms](#critic-based-algorithms)
- [Critic-Free Algorithms](#critic-free-algorithms)
- [Off-policy Optimization](#off-policy-optimization)
- [Off-policy Optimization (Exp replay)](#off-policy-optimization-exp-replay)
- [Regularization Objectives](#regularization-objectives)
- [Sampling Strategy](#sampling-strategy)
- [Dynamic and Structured Sampling](#dynamic-and-structured-sampling)
- [Sampling Hyper-Parameters](#sampling-hyper-parameters)
- [Training Resource](#training-resource)
- [Static Corpus (Code)](#static-corpus-code)
- [Static Corpus (STEM)](#static-corpus-stem)
- [Static Corpus (Math)](#static-corpus-math)
- [Static Corpus (Agent)](#static-corpus-agent)
- [Static Corpus (Mix)](#static-corpus-mix)
- [Dynamic Environment (Rule-based)](#dynamic-environment-rule-based)
- [Dynamic Environment (Code-based)](#dynamic-environment-code-based)
- [Dynamic Environment (Game-based)](#dynamic-environment-game-based)
- [Dynamic Environment (Model-based)](#dynamic-environment-model-based)
- [Dynamic Environment (Ensemble-based)](#dynamic-environment-ensemble-based)
- [RL Infrastructure (Primary)](#rl-infrastructure-primary)
- [RL Infrastructure (Secondary)](#rl-infrastructure-secondary)
- [Applications](#applications)
- [Coding Agent](#coding-agent)
- [Search Agent](#search-agent)
- [Browser-Use Agent](#browser-use-agent)
- [DeepResearch Agent](#deepresearch-agent)
- [GUI\&Computer Agent](#guicomputer-agent)
- [Recommendation Agent](#recommendation-agent)
- [Agent (Others)](#agent-others)
- [Code Generation](#code-generation)
- [Software Engineering](#software-engineering)
- [Multimodal Understanding](#multimodal-understanding)
- [Multimodal Generation](#multimodal-generation)
- [Robotics Tasks](#robotics-tasks)
- [Multi-Agent Systems](#multi-agent-systems)
- [Scientific Tasks](#scientific-tasks)
- [π Acknowledgment](#-acknowledgment)
- [β¨ Star History](#-star-history)
## πΊοΈ Overview
Our survey provides a comprehensive examination of **Reinforcement Learning for Large Reasoning Models**.
