SAPO

A PPO-style Agentic RL framework without a separate critic model and multiple rollouts

SAPO is a compute- and memory-efficient PPO-style framework designed for long-horizon LLM agentic training. By learning the policy and value functions within a single causal language model, SAPO entirely eliminates PPO's separate critic model and the multiple rollouts typically required by group-relative methods. Consequently, with identical sampling budgets, SAPO matches PPO's performance and substantially outperforms GRPO, while achieving a significant reduction in resource consumption.