DPO Pipeline

DPO eliminates the separate reward model AND the PPO optimizer entirely by using a mathematically equivalent closed-form loss. Quality of CHOSEN responses is the primary driver — focus annotation budget there, not on chosen/rejected balance.