Exploratory Preference Optimization: Harnessing Implicit Q* -Approximation for Sample-Efficient RLHF - Xie, Tengyang; Foster, Dylan; Krishnamurthy, Akshay; Rosset, Corby; Awadallah, Ahmed H; Rakhlin, Alexander (International Conference on Learning Representations, 2025)