👨‍🎓 About Me

Hi! I am an incoming PhD at BAIR, UC Berkeley. Prior to that, I worked as a research intern at AIR-DREAM Lab of Institute for AI Industry Research (AIR), Tsinghua University, advised by Prof. Xianyuan Zhan. I obtained B.S. and M.S. in 2022 and 2025 from the Department of Automation, Tsinghua University. My research interest lies in reinforcement learning and cross-domain policy adaptation (i.e. sim-to-real) for embodied agents.

🔥 News

• [2025.05] Invited talk about LBP (ICML 2025) at RLChina Academic Seminar!
• [2025.05] One paper (LBP) is accepted by ICML 2025!
• [2025.01] One paper (H2O+) is accepted by ICRA 2025! One paper (PSEC) is accepted by ICLR 2025!
• [2025.01] Guest Speaker sharing RSP at AI TIME AAAI Forum. [video(Chinese)]
• [2024.12] One paper (RSP) is accepted by AAAI 2025 as Oral Presentation!
• [2024.10] Two papers (xTED, RSP) are accepted by NeurIPS 2024 Workshop on Open-World Agents.
• [2024.05] One paper (DecisionNCE) is accepted by ICML 2024!
• [2024.04] One Survey paper on Cross-Domain Embodied Policy Transfer is accepted by IJCAI 2024!
• [2024.01] One paper (CLIC) is accepted by ICRA 2024!
• [2023.10] Two papers (CLIC, SDM) are accepted by NeurIPS 2023 Symposium on ML4AD and ALOE Workshop!
• [2023.03] One paper ((Re)$^2$H2O) is accepted by IEEE IV 2023!
• [2023.02] Guest Speaker at 2023 AI TIME Youth PhD Talk Forum.
• [2022.11] Invited talk about our NeurIPS paper (H2O) at RLChina Academic Seminar! [video]
• [2022.11] Invited talk about our NeurIPS paper (H2O) at DI Lab’s group meeting!
• [2022.10] One paper is accepted by NeurIPS 2022 RL4RealLife Workshop and ML4AD Workshop!
• [2022.09] One paper (H2O) is accepted by NeurIPS 2022 as Spotlight!
• [2022.09] One paper (DMIL) is accepted by CoRL 2022!

📝 Publications

Preprints, Codebases and Workshop Papers

• Haoyi Niu$^*$, Qimao Chen$^*$, Tenglong Liu, Jianxiong Li, Guyue Zhou, Yi Zhang, Jianming Hu and Xianyuan Zhan, xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing, NeurIPS Workshop on Open-World Agents, 2024. [Code][Website]

• Xiangyu Zhu$^*$, Jianxiong Li$^*$, Haoyi Niu$^*$, Yinan Zheng$^*$, Peng Cheng$^*$, Wenjia Zhang$^*$, Haoran Xu$^*$ and Xianyuan Zhan, D2C: A Data-Driven Control Library Based on Reinforcement Learning, Paper coming soon, 2023. [Code][Documentation]

• Haoyi Niu$^*$, Qimao Chen$^*$, Yingyue Li and Jianming Hu, Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving, NeurIPS 2023 Symposium on Machine Learning for Autonomous Driving & Agent Learning in Open-Endedness Workshop, 2023. [Code]

• Guan Wang$^*$, Haoyi Niu$^*$, Desheng Zhu, Jianming Hu, Xianyuan Zhan and Guyue Zhou, A Versatile and Efficient Reinforcement Learning Approach for Autonomous Driving, NeurIPS Machine Learning for Autonomous Driving Workshop, NeurIPS RL for RealLife Workshop, 2022. [Video][OneRL Code][NoGap Benchmark]

Conference Proceedings

• Dongxiu Liu$^*$, Haoyi Niu$^*$, Zhihao Wang, Jinliang Zheng, Yinan Zheng, Zhonghong Ou, Jianming Hu, Jianxiong Li, Xianyuan Zhan, Efficient Robotic Policy Learning via Latent Space Backward Planning, International Conference on Machine Learning (ICML 2025). [Code][Website]
• Haoyi Niu$^*$, Tianying Ji$^*$, Bingqi Liu, Haocheng Zhao, Xiangyu Zhu, Jianying Zheng, Pengfei Huang, Guyue Zhou, Jianming Hu and Xianyuan Zhan, H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps, IEEE International Conference on Robotics and Automation (ICRA 2025). [Code][Wbsite]
• Tenglong Liu$^*$, Jianxiong Li$^*$, Yinan Zheng, Haoyi Niu, Yixing Lan, Xin Xu and Xianyuan Zhan, Skill Expansion and Composition in Parameter Space, International Conference on Learning Representations (ICLR 2025).
• Guan Wang$^*$, Haoyi Niu$^*$, Jianxiong Li, Li Jiang, Jianming Hu and Xianyuan Zhan, Are Expressive Models Truly Necessary for Offline RL?, Annual AAAI Conference on Artificial Intelligence (AAAI 2025), Oral Presentation. [Code][Twitter]
• Jianxiong Li, Jinliang Zheng, Yinan Zheng, Liyuan Mao, Xiao Hu, Sijie Cheng, Haoyi Niu, Jihao Liu, Yu Liu, Jingjing Liu, Ya-Qin Zhang and Xianyuan Zhan, DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning, International Conference on Machine Learning (ICML 2024). [Website][Code]
• Haoyi Niu, Jianming Hu, Guyue Zhou and Xianyuan Zhan, A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents, International Joint Conference on Artificial Intelligence (IJCAI 2024), Oral Presentation. [Repo]
• Haoyi Niu$^*$, Yizhou Xu$^*$, Xingjian Jiang and Jianming Hu, Continual Driving Policy Optimization with Closed-Loop Individualized Curricula, IEEE International Conference on Robotics and Automation (ICRA 2024). [Code]
• Haoyi Niu$^*$, Kun Ren$^*$, Yizhou Xu, Ziyuan Yang, Yichen Lin, Yi Zhang and Jianming Hu, (Re)$^2$H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning, IEEE Intelligent Vehicles Symposium 2023 (IV 2023). [Code]
• Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming Hu and Xianyuan Zhan, When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning, Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022), Spotlight. [Video][Code][Spotlight]
• Wenjia Zhang, Haoran Xu, Haoyi Niu, Peng Cheng, Ming Li, Heming Zhang, Guyue Zhou and Xianyuan Zhan, Discriminator-Guided Model-Based Offline Imitation Learning, 6th Conference on Robot Learning (CoRL 2022).
• Haoyi Niu, Jianming Hu, Zheyu Cui and Yi Zhang, Tactical Decision Making for Emergency Vehicles Based on A Combinational Learning Method, the 20th and 21st Joint COTA International Conference of Transportation Professionals (CICTP 2020-21), Best Paper Award.
• Haoyi Niu, Jianming Hu, Zheyu Cui and Yi Zhang, DR$^2$L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain Randomization Reinforcement Learning, the 5th International Conference on Computer Science and Application Engineering (CSAE 2021), Oral Presentation.
• Qingyu Song, Ruibo Ming, Jianming Hu, Haoyi Niu and Mingyang Gao, Graph Attention Convolutional Network: Spatiotemporal Modeling for Urban Traffic Prediction, IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC 2020).

🎖 Honors and Awards

• [2025.06] Beijing Outstanding Graduate Award
• [2025.06] Tsinghua Outstanding Master Dissertation (top 5%)
• [2025.06] Tsinghua Outstanding Master Graduate Award (top 1%)
• [2025.06] Outstanding Master Graduate Award of Department of Automation, Tsinghua University
• [2025.01] AAAI 2025 Oral Presentation Award
• [2024.11] Tsinghua First-Class Comprehensive Excellence Award for Graduate Students (Fang Chongzhi Development Fund Scholarship)
• [2024.01] Zheng Weimin Scholarship
• [2023.10] Tsinghua First-Class Comprehensive Excellence Award for Graduate Students (Lingjun Investment Scholarship)
• [2022.12] NeurIPS 2022 Spotlight Award
• [2022.06] Tsinghua Outstanding Bachelor Dissertation (top 5%)
• [2022.06] Tsinghua Excellent Bachelor Graduate Award (top 10%)
• [2022.06] Outstanding Bachelor Graduate Award of Department of Automation, Tsinghua University
• [2021.12] Best Paper Award at COTA International Conference of Transportation Professionals (CICTP)
• [2021.10] Tsinghua Scientific and Technological Innovation Excellence Award
• [2021.09] Tsinghua “Future Scholar” Scientific Research Grant (9/3850 in Tsinghua Class of 2022)
• [2021.04] Second Prize in Tsinghua “Challenge Cup” Academic Science and Technology Competition
• [2020.12] First Prize in Tsinghua Excellent Student Research Training (SRT) Program (top 1%, 12/1250)
• [2020.10] Tsinghua Comprehensive Excellence Award (top 10%, 15/179)
• [2020.10] Fang Chongzhi Scholarship (highest scholarship for juniors in Dept. of Automation, 1/179)
• [2019.12] First Prize in Group A (Non-Physics Major) of the National Collegiate Physics Competition

📖 Educations

• 2022.08 - 2025.06, Master, Department of Automation, Tsinghua University, Beijing, China.
• 2018.08 - 2022.06, Undergraduate, Department of Automation, Tsinghua Univeristy, Beijing, China.
• 2015.08 - 2018.06, High School Attached to Northeast Normal University, Changchun, Jilin, China.

💻 Internships

• 2021.03 - 2025.06, AIR-DREAM Lab of Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China. (Advisor: Prof.Xianyuan Zhan)

🧑‍🎨 Services

Reviewer for Conferences (ITSC, ICRA, IROS, IJCAI, NeurIPS, ICML) and Journals (IEEE Robotics and Automation Letters, IEEE Transactions on Intelligent Vehicles, Transportation Research Part C).