👨‍🎓 About Me

Hi! I am currently a master student at Intelligent Transportation Lab, Institute of Systems Engineering, Department of Automation, Tsinghua University, advised by Prof. Jianming Hu. Additionally, I work as a research intern at AIR-DREAM Lab of Institute for AI Industry Research (AIR), Tsinghua University, advised by Prof. Xianyuan Zhan. I am also fortunate to work closely with Prof. Dorsa Sadigh at Stanford. I got my bachelor’s degree in June 2022 from the Department of Automation, Tsinghua University.

My research interest broadly lies in advanced data-driven learning theory and algorithms on decision making and optimization, such as reinforcement learning (RL), as well as their promising applications on autonomous driving and robotics. However, over recent years, criticism against RL continues to pour regarding its limited real-world applicability. Specifically, I therefore pay much attention to bridging the complex and intractable sim-to-real gaps that potentially deteriorate RL policies, seeking more practical solutions for real-world deployment.

🔥 News

• [2024.04] One paper is accepted by IJCAI 2024!
• [2024.01] One paper is accepted by ICRA 2024!
• [2023.10] Two papers are accepted by NeurIPS 2023 Symposium on ML4AD and ALOE Workshop!
• [2023.03] One paper is accepted by IEEE IV 2023!
• [2023.02] Guest Speaker at 2023 AI TIME Youth PhD Talk Forum.
• [2022.11] Invited talk about our NeurIPS paper (H2O) at RLChina Academic Seminar! [video]
• [2022.11] Invited talk about our NeurIPS paper (H2O) at DI Lab’s group meeting!
• [2022.10] One paper is accepted by NeurIPS 2022 RL4RealLife Workshop and ML4AD Workshop!
• [2022.09] One paper is accepted by NeurIPS 2022!
• [2022.09] One paper is accepted by CoRL 2022!

📝 Publications

Preprints and Codebases

• Jianxiong Li, Jinliang Zheng, Yinan Zheng, Liyuan Mao, Xiao Hu, Sijie Cheng, Haoyi Niu, Jihao Liu, Yu Liu, Jingjing Liu, Ya-Qin Zhang and Xianyuan Zhan, DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning, Under Review, 2024.

• Xiangyu Zhu$^*$, Jianxiong Li$^*$, Haoyi Niu$^*$, Yinan Zheng$^*$, Peng Cheng$^*$, Wenjia Zhang$^*$, Haoran Xu$^*$ and Xianyuan Zhan, D2C: A Data-Driven Control Library Based on Reinforcement Learning, Paper coming soon, 2023. [Code][Documentation]

• Haoyi Niu$^*$, Tianying Ji$^*$, Bingqi Liu, Haocheng Zhao, Xiangyu Zhu, Jianying Zheng, Pengfei Huang, Guyue Zhou, Jianming Hu and Xianyuan Zhan, H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps, ICLR Workshop on Data-centric Machine Learning Research (DMLR), 2024.

• Haoyi Niu$^*$, Qimao Chen$^*$, Yingyue Li and Jianming Hu, Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving, NeurIPS 2023 Symposium on Machine Learning for Autonomous Driving & Agent Learning in Open-Endedness Workshop, 2023. [Code]

• Guan Wang$^*$, Haoyi Niu$^*$, Desheng Zhu, Jianming Hu, Xianyuan Zhan and Guyue Zhou, A Versatile and Efficient Reinforcement Learning Approach for Autonomous Driving, NeurIPS Machine Learning for Autonomous Driving Workshop, NeurIPS RL for RealLife Workshop, 2022. [Video][OneRL Code][NoGap Benchmark]

Conference Proceedings

• Haoyi Niu, Jianming Hu, Guyue Zhou and Xianyuan Zhan, A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents, International Joint Conference on Artificial Intelligence (IJCAI 2024). [Repo]
• Haoyi Niu$^*$, Yizhou Xu$^*$, Xingjian Jiang and Jianming Hu, Continual Driving Policy Optimization with Closed-Loop Individualized Curricula, IEEE International Conference on Robotics and Automation (ICRA 2024). [Code]
• Haoyi Niu$^*$, Kun Ren$^*$, Yizhou Xu, Ziyuan Yang, Yichen Lin, Yi Zhang and Jianming Hu, (Re)$^2$H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning, IEEE Intelligent Vehicles Symposium 2023 (IV 2023). [Code]
• Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming Hu and Xianyuan Zhan, When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning, Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022), Spotlight. [Video][Code]
• Wenjia Zhang, Haoran Xu, Haoyi Niu, Peng Cheng, Ming Li, Heming Zhang, Guyue Zhou and Xianyuan Zhan, Discriminator-Guided Model-Based Offline Imitation Learning, 6th Conference on Robot Learning (CoRL 2022).
• Haoyi Niu, Jianming Hu, Zheyu Cui and Yi Zhang, Tactical Decision Making for Emergency Vehicles Based on A Combinational Learning Method, the 20th and 21st Joint COTA International Conference of Transportation Professionals (CICTP 2020-21), Best Paper Award.
• Haoyi Niu, Jianming Hu, Zheyu Cui and Yi Zhang, DR$^2$L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain Randomization Reinforcement Learning, the 5th International Conference on Computer Science and Application Engineering (CSAE 2021).
• Qingyu Song, Ruibo Ming, Jianming Hu, Haoyi Niu and Mingyang Gao, Graph Attention Convolutional Network: Spatiotemporal Modeling for Urban Traffic Prediction, IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC 2020).

Dissertation

• Haoyi Niu, Robot Policy Learning with Imperfect Simulation: Effective Sim-to-Real Methods and Implementations (in Chinese), 2022, Outstanding Graduate Dissertation Award.

🎖 Honors and Awards

• [2024.01] Weimin Zheng Scholarship
• [2023.10] Tsinghua First-Class Comprehensive Excellence Award for Graduate Students (Lingjun Investment Scholarship)
• [2022.06] Tsinghua Outstanding Graduate Dissertation (top 5%)
• [2022.06] Tsinghua Outstanding Graduate Award (top 10%)
• [2022.06] Excellent Graduate Award of Department of Automation, Tsinghua University
• [2021.12] Best Paper Award at COTA International Conference of Transportation Professionals (CICTP)
• [2021.09] Tsinghua “Future Scholar” Scientific Research Grant (9/3850 in Tsinghua Class of 2022)
• [2021.04] Second Prize in Tsinghua “Challenge Cup” Academic Science and Technology Competition
• [2020.12] First Prize in Tsinghua Excellent Student Research Training (SRT) Program (top 1%, 12/1250)
• [2020.10] Tsinghua Comprehensive Excellence Award (top 10%, 15/179)
• [2020.10] Fang Chongzhi Scholarship (highest scholarship for juniors in Dept. of Automation, 1/179)
• [2019.12] First Prize in Group A (Non-Physics Major) of the National Collegiate Physics Competition

📖 Educations

• 2022.08 - Present, Master, Department of Automation, Tsinghua University, Beijing, China.
• 2018.08 - 2022.06, Undergraduate, Department of Automation, Tsinghua Univeristy, Beijing, China.
• 2015.08 - 2018.06, High School Attached to Northeast Normal University, Changchun, Jilin, China.

💻 Internships

• 2021.03 - Present, AIR-DREAM Lab of Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China. (Advisor: Prof.Xianyuan Zhan)
• 2023.06 - Present, Intelligent and Interactive Autonomous Systems Group (ILIAD), Stanford University, Stanford, USA. (Advisor: Dr. Yuchen Cui and Prof. Dorsa Sadigh)

🧑‍🎨 Services

Reviewer for Conferences (ITSC, ICRA, IROS, IJCAI, NeurIPS, ICML) and Journals (IEEE Transactions on Intelligent Vehicles, Transportation Research Part C).