AI Architecture & Systems

RL Frameworks and Source-Code Studies

5.31.1verl: Initialization, Rollout, and Training Workflow#

5.31.2slime: Rollout-First Design and Backend Integration#

5.31.3OpenRLHF and Ray-Based Actor Management#

5.31.4AReaL and Asynchronous RL Training#

5.31.5TRL and Lightweight Post-Training Workflows#

5.31.6Weight Update, Memory Sleep/Wake, and Resource Reuse#

5.31.7FP8/INT4, Speculative Decoding, and Multi-Turn Tasks in RL#