GRPO leaves the chatbot and starts fixing OCR, while DPO and GRPO get debugged from the inside