Why is Your Language Model a Poor Implicit Reward Model? - Razin, Noam; Lin, Yong; Yao, Jiarui; Arora, Sanjeev (International Conference on Learning Representations, 2026)