👨‍🎓 About Me

Hi! I am currently a master student at Intelligent Transportation Lab, Institute of Systems Engineering, Department of Automation, Tsinghua University, advised by Prof. Jianming Hu. Additionally, I work as a research intern at AIR-DREAM Lab of Institute for AI Industry Research (AIR), Tsinghua University, advised by Prof. Xianyuan Zhan. I got my bachelor’s degree in June 2022 from the Department of Automation, Tsinghua University.

My research interest broadly lies in advanced data-driven learning theory and algorithms on decision making and optimization, such as reinforcement learning (RL), as well as their promising applications on autonomous driving and robotics. Specifically, I pay much attention to bridging the complex and intractable sim-to-real/embodiment gaps that potentially deteriorate policies, seeking more practical solutions for building real-world embodied agents.

🔥 News

• [2024.10] Two papers (xTED, RSP) are accepted by NeurIPS 2024 Workshop on Open-World Agents.
• [2024.05] One paper (DecisionNCE) is accepted by ICML 2024!
• [2024.04] One paper is accepted by IJCAI 2024!
• [2024.01] One paper (CLIC) is accepted by ICRA 2024!
• [2023.10] Two papers (CLIC, SDM) are accepted by NeurIPS 2023 Symposium on ML4AD and ALOE Workshop!
• [2023.03] One paper ((Re)$^2$H2O) is accepted by IEEE IV 2023!
• [2023.02] Guest Speaker at 2023 AI TIME Youth PhD Talk Forum.
• [2022.11] Invited talk about our NeurIPS paper (H2O) at RLChina Academic Seminar! [video]
• [2022.11] Invited talk about our NeurIPS paper (H2O) at DI Lab’s group meeting!
• [2022.10] One paper is accepted by NeurIPS 2022 RL4RealLife Workshop and ML4AD Workshop!
• [2022.09] One paper (H2O) is accepted by NeurIPS 2022!
• [2022.09] One paper (DMIL) is accepted by CoRL 2022!

📝 Publications

Preprints, Codebases and Workshop Papers

• Haoyi Niu$^*$, Qimao Chen$^*$, Tenglong Liu, Jianxiong Li, Guyue Zhou, Yi Zhang, Jianming Hu and Xianyuan Zhan, xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing, NeurIPS Workshop on Open-World Agents, 2024. [Code][Website]

• Tenglong Liu$^*$, Jianxiong Li$^*$, Yinan Zheng, Haoyi Niu, Yixing Lan, Xin Xu and Xianyuan Zhan, Skill Expansion and Composition in Parameter Space, Under Review, 2024.

• Guan Wang$^*$, Haoyi Niu$^*$, Jianxiong Li, Li Jiang, Jianming Hu and Xianyuan Zhan, Are Expressive Models Truly Necessary for Offline RL?, NeurIPS Workshop on Open-World Agents, 2024. [Code]

• Xiangyu Zhu$^*$, Jianxiong Li$^*$, Haoyi Niu$^*$, Yinan Zheng$^*$, Peng Cheng$^*$, Wenjia Zhang$^*$, Haoran Xu$^*$ and Xianyuan Zhan, D2C: A Data-Driven Control Library Based on Reinforcement Learning, Paper coming soon, 2023. [Code][Documentation]

• Haoyi Niu$^*$, Tianying Ji$^*$, Bingqi Liu, Haocheng Zhao, Xiangyu Zhu, Jianying Zheng, Pengfei Huang, Guyue Zhou, Jianming Hu and Xianyuan Zhan, H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps, ICLR Workshop on Data-centric Machine Learning Research (DMLR), 2024.

• Haoyi Niu$^*$, Qimao Chen$^*$, Yingyue Li and Jianming Hu, Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving, NeurIPS 2023 Symposium on Machine Learning for Autonomous Driving & Agent Learning in Open-Endedness Workshop, 2023. [Code]

• Guan Wang$^*$, Haoyi Niu$^*$, Desheng Zhu, Jianming Hu, Xianyuan Zhan and Guyue Zhou, A Versatile and Efficient Reinforcement Learning Approach for Autonomous Driving, NeurIPS Machine Learning for Autonomous Driving Workshop, NeurIPS RL for RealLife Workshop, 2022. [Video][OneRL Code][NoGap Benchmark]

Conference Proceedings

• Jianxiong Li, Jinliang Zheng, Yinan Zheng, Liyuan Mao, Xiao Hu, Sijie Cheng, Haoyi Niu, Jihao Liu, Yu Liu, Jingjing Liu, Ya-Qin Zhang and Xianyuan Zhan, DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning, International Conference on Machine Learning (ICML 2024). [Website][Code]
• Haoyi Niu, Jianming Hu, Guyue Zhou and Xianyuan Zhan, A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents, International Joint Conference on Artificial Intelligence (IJCAI 2024), Oral Presentation. [Repo]
• Haoyi Niu$^*$, Yizhou Xu$^*$, Xingjian Jiang and Jianming Hu, Continual Driving Policy Optimization with Closed-Loop Individualized Curricula, IEEE International Conference on Robotics and Automation (ICRA 2024). [Code]
• Haoyi Niu$^*$, Kun Ren$^*$, Yizhou Xu, Ziyuan Yang, Yichen Lin, Yi Zhang and Jianming Hu, (Re)$^2$H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning, IEEE Intelligent Vehicles Symposium 2023 (IV 2023). [Code]
• Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming Hu and Xianyuan Zhan, When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning, Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS 2022), Spotlight. [Video][Code]
• Wenjia Zhang, Haoran Xu, Haoyi Niu, Peng Cheng, Ming Li, Heming Zhang, Guyue Zhou and Xianyuan Zhan, Discriminator-Guided Model-Based Offline Imitation Learning, 6th Conference on Robot Learning (CoRL 2022).
• Haoyi Niu, Jianming Hu, Zheyu Cui and Yi Zhang, Tactical Decision Making for Emergency Vehicles Based on A Combinational Learning Method, the 20th and 21st Joint COTA International Conference of Transportation Professionals (CICTP 2020-21), Best Paper Award.
• Haoyi Niu, Jianming Hu, Zheyu Cui and Yi Zhang, DR$^2$L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain Randomization Reinforcement Learning, the 5th International Conference on Computer Science and Application Engineering (CSAE 2021).
• Qingyu Song, Ruibo Ming, Jianming Hu, Haoyi Niu and Mingyang Gao, Graph Attention Convolutional Network: Spatiotemporal Modeling for Urban Traffic Prediction, IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC 2020).

Dissertation

• Haoyi Niu, Robot Policy Learning with Imperfect Simulation: Effective Sim-to-Real Methods and Implementations (in Chinese), 2022, Outstanding Graduate Dissertation Award.

🎖 Honors and Awards

• [2024.01] Weimin Zheng Scholarship
• [2023.10] Tsinghua First-Class Comprehensive Excellence Award for Graduate Students (Lingjun Investment Scholarship)
• [2022.06] Tsinghua Outstanding Graduate Dissertation (top 5%)
• [2022.06] Tsinghua Outstanding Graduate Award (top 10%)
• [2022.06] Excellent Graduate Award of Department of Automation, Tsinghua University
• [2021.12] Best Paper Award at COTA International Conference of Transportation Professionals (CICTP)
• [2021.09] Tsinghua “Future Scholar” Scientific Research Grant (9/3850 in Tsinghua Class of 2022)
• [2021.04] Second Prize in Tsinghua “Challenge Cup” Academic Science and Technology Competition
• [2020.12] First Prize in Tsinghua Excellent Student Research Training (SRT) Program (top 1%, 12/1250)
• [2020.10] Tsinghua Comprehensive Excellence Award (top 10%, 15/179)
• [2020.10] Fang Chongzhi Scholarship (highest scholarship for juniors in Dept. of Automation, 1/179)
• [2019.12] First Prize in Group A (Non-Physics Major) of the National Collegiate Physics Competition

📖 Educations

• 2022.08 - Present, Master, Department of Automation, Tsinghua University, Beijing, China.
• 2018.08 - 2022.06, Undergraduate, Department of Automation, Tsinghua Univeristy, Beijing, China.
• 2015.08 - 2018.06, High School Attached to Northeast Normal University, Changchun, Jilin, China.

💻 Internships

• 2021.03 - Present, AIR-DREAM Lab of Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China. (Advisor: Prof.Xianyuan Zhan)

🧑‍🎨 Services

Reviewer for Conferences (ITSC, ICRA, IROS, IJCAI, NeurIPS, ICML) and Journals (IEEE Transactions on Intelligent Vehicles, Transportation Research Part C).