GeezerVison

GeezerVison is showing www.mengdixu.me — Mengdi Xu

Home News Publications Service Mentoring

Mengdi Xu

Hi! I am an assistant professor at the Institute for Interdisciplinary Information Sciences (IIIS) at Tsinghua University. Previously, I was a postdoc at Stanford Vision and Learning Lab (SVL) working with Prof. Jiajun Wu and Prof. Fei-Fei Li. I received my PhD from Carnegie Mellon University with Best PhD Dissertation Award, advised by Prof. Ding Zhao (SafeAI Lab). My thesis is about building adaptable generalist robots. I have also spent wonderful summers at Google DeepMind Robotics, MIT-IBM Watson AI Lab, and Toyota Research Institute. I received a B.E. from Tsinghua University.

Email / CV (Mar 2024) / Google Scholar / Github / X

I am actively recruiting students to join my lab at Tsinghua IIIS and am also open to research collaborations. If you are interested, please feel free to reach out!

Prospective Students & Collaborations

Thank you for your interest in our lab! We welcome highly motivated undergraduates, graduate students, and visiting scholars to join us and collaborate. We are a highly interdisciplinary lab seeking students from diverse backgrounds, including robot hardware design, large-scale model training, machine learning, control, and robotics.
For undergraduates and graduate students, please complete this form and send me an email.
For potential collaborators, feel free to email me directly with the projects you are interested in.

Research

I am broadly interested in building scalable, adaptable, and reliable robots that seamlessly interact with humans in daily activities. Here are some research highlights:

In-context robot learning with few-shot demonstrations and non-expert data.
Unsupervised learning to discover task/skill structures in dynamics, visual geometries, and agent policies.
Distributionally robust RL that balances performance and robustness when facing task uncertainties.

Group

PhD students:

News

2026/10 - Invited talk at IROS 2026 IROS scale-infra workshop!
2026/09 - CATok, our causal action tokenizer, got accepted to NeurIPS 2026!
2026/09 - DEX-X, MIRROR and Humanoid Badminton got accepted to CoRL 2026!
2026/08 - We are organizing the Continually Self-Improving Robots workshop at CoRL 2026. Join us for discussions on robot self-improvement and in-context robot learning!
2026/06 - Invited talk at ICRA 2026 Workshop on “Semantics for Reliable Robot Autonomy: From Environment Understanding and Reasoning to Safe Interactions”!
2026/02 - We released CoVer-VLA, a test-time scaling framework for VLA.
2026/02 - Honored to serve as a Program Committee (PC) member for RSS Pioneers 2026.
2026/01 - MoMaGen and ROSETTA got accepted to ICLR 2026!
2025/10 - MoMaGen released. Check out how MoMaGen generates data for long-horizon bimanual mobile manipulation tasks.
2025/09 - I joined Tsinghua IIIS as a tenure-track Assistant Professor.
2025/06 - ROSETTA won the best paper award at the RSS 2025 Workshop on Continual Robot Learning from Humans (CRLH).
2024/08 - Functional Optimal Transport got accepted to JMLR.
2024/07 - Invited talk at CMU LeCAR Lab.
2024/06 - Joined Stanford Vision and Learning Lab (SVL) as a postdoctoral researcher.
2024/05 - We are organizing the RSS Workshop on Lifelong Robot Learning: Generalization, Adaptation, and Deployment with Large Models and RSS Pioneers 2024 Workshop. See you in Delft!

Publications

( / / )

Rethinking Causal Action Tokenization with Condition Annealing in Flow Matching
Chenyu Zhang*, Yuhang Cao*, Yingxi Lu, Daru Du, Jing Shao, Jiajun Liu, Ruoqu Chen, Liu Cao, Yicheng Liu, Hang Zhao, Mengdi Xu
NeurIPS 2026.
[project]

WEAVE: Learning Whole-Body Dexterous Loco-Manipulation from Human–Object Interactions
Liu Cao, Xingze Wu, Jingzhi Cui, Botian Xu, Mingzhi Pei, Ruoqu Chen, Mengdi Xu
arXiv preprint, 2026.
[project] [paper]

Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction
Ruoqu Chen, Feixiang Ruan, Liu Cao, Zihao Wang, Botian Xu, Shiqin Tong, Jiajun Liu, Wanli Xing, Kaifeng Zhang, Mengdi Xu
CoRL 2026. RSS 2026 Workshop on Dexterous Manipulation (spotlight).
[project]

Uni-VLaT: Whole-Body Tactile Adaptation of VLA Policies for Humanoid Loco-Manipulation
Zihao Wang*, Shutong Liu*, Siqi Zheng, Liu Cao, Ruoqu Chen, Rundong Liu, Yanchao Yang, Mengdi Xu
arXiv preprint, 2026.
[project] [paper]

Humanoid Badminton: Learning Dynamic Racket Skills from Limited Human Motion Data
Jingzhi Cui, Zhexiong Wang, Bangjie Xu, Pengyu Zhao, Youyuan Li, Zhi Su, Peng Ren, Mengdi Xu, Chao Yu, Yi Wu, Luyang Wang, Zhongyu Li
CoRL 2026.
[project] [paper]

HuMiT: Low-Latency Whole-Body Humanoid Teleoperation via Minimal Reference Tracking
Liu Cao, Botian Xu, Ruoqu Chen, Jiajun Liu, Mingzhi Pei, Zihao Wang, Zhongyu Li, Mengdi Xu
Preprint, 2026.
[project] [paper] [code]

What Do VLAs Actually Learn through In-Context Failure Conditioning?
Jiajun Liu, Jieming Li, Zi Zhuang, Hang Yu, Qingli Chen, Liu Cao, Yingxi Lu, Ruoqu Chen, Yuhang Cao, Chenyu Zhang, Yankai Lin, Mengdi Xu
Preprint. CVPR 2026 Workshop on 3D-LLM/VLA.
[project]

Data Generation

MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation
Chengshu Li*, Mengdi Xu*, Arpit Bahety*, Hang Yin*, Yunfan Jiang, Huang Huang, Josiah Wong, Sujay Garlanka, Cem Gokmen, Ruohan Zhang, Weiyu Liu, Jiajun Wu, Roberto Martín-Martín, Fei-Fei Li
ICLR 2026.
[paper] [website] [code]

ROSETTA: Constructing Code-Based Reward from Unconstrained Language Preference
Sanjana Srivastava*, Kangrui Wang*, Yung-Chieh Chan*, Tianyuan Dai, Manling Li, Ruohan Zhang, Mengdi Xu, Jiajun Wu, Fei-Fei Li
ICLR 2026.
[website]

Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment
Jacky Kwok*, Xilun Zhang*, Mengdi Xu, Yuejiang Liu†, Azalia Mirhoseini†, Chelsea Finn†, Marco Pavone†
ECCV 2026. † equal advising.
[paper] [website]

Creative Robot Tool Use with Large Language Models
Mengdi Xu*, Peide Huang*, Wenhao Yu*, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, Ding Zhao
CoRL 2023 Workshop on Language and Robot Learning: Language as Grounding
[paper] [webpage] [MLD Blog] [TechXplore]

Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications
Xilun Zhang*, Shiqi Liu*, Peide Huang, William Han, Yiqi Lyu, Mengdi Xu, Ding Zhao
IEEE Robotics and Automation Letters (RA-L), 2025
Abridged in AAAI 2025 Workshop on Multi-Agent AI in the Real World (oral presentation)
[paper] [website]

Hyper-Decision Transformer for Efficient Online Policy Adaptation
Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, Chuang Gan
ICLR, 2023
[paper] [webpage]

Prompting Decision Transformer for Few-shot Policy Generalization
Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Josh Tenenbaum, Chuang Gan
ICML, 2022
[paper] [webpage] [code]

Adaptive Online Replanning with Diffusion Models
Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit-Yan Yeung, Chuang Gan
NeurIPS, 2023
[paper] [webpage]

Continual Vision-based Reinforcement Learning with Group Symmetries
Shiqi Liu*, Mengdi Xu*, Peide Huang, Yongkang Liu, Kentaro Oguchi, Ding Zhao
CoRL, 2023 (oral)
[paper] [webpage] [code]

Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptation
Peide Huang, Mengdi Xu, Jiacheng Zhu, Laixi Shi, Fei Fang, Ding Zhao,
NeurIPS, 2022
[paper]

Functional Optimal Transport: Map Estimation and Domain Adaptation for Functional data
Jiacheng Zhu*, Aritra Guha*, Dat Do*, Mengdi Xu, XuanLong Nguyen, Ding Zhao
JMLR: Journal of Machine Learning Research
AAAI OT-SDM 2022 workshop (spotlight)
[paper] [code]

Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes
Mengdi Xu, Wenhao Ding, Jiacheng Zhu, Zuxin Liu, Baiming Chen, Ding Zhao
NeurIPS, 2020
[paper] [video] [code] [bibtex]

Learning from Human

Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction
Ruoqu Chen, Feixiang Ruan, Liu Cao, Zihao Wang, Botian Xu, Shiqin Tong, Jiajun Liu, Wanli Xing, Kaifeng Zhang, Mengdi Xu
RSS 2026 Workshop on Dexterous Manipulation (spotlight).
[project]

Foundation Model

Few-shot Generalization

Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment
Jacky Kwok*, Xilun Zhang*, Mengdi Xu, Yuejiang Liu†, Azalia Mirhoseini†, Chelsea Finn†, Marco Pavone†
In submission, 2026. † equal advising.
[paper] [website]

Hyper-Decision Transformer for Efficient Online Policy Adaptation
Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, Chuang Gan
The Eleventh International Conference on Learning Representations (ICLR), 2023
[paper] [webpage]

Prompting Decision Transformer for Few-shot Policy Generalization
Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Josh Tenenbaum, Chuang Gan
Thirty-ninth International Conference on Machine Learning (ICML), 2022
[paper] [webpage] [code]

Efficient Adaptation

Continual Vision-based Reinforcement Learning with Group Symmetries
Shiqi Liu*, Mengdi Xu*, Peide Huang, Yongkang Liu, Kentaro Oguchi, Ding Zhao
Conference on Robot Learning (CoRL), 2023 (oral)
[paper] [webpage] [code]

Robust and Safe Robot Learning

Scalable Safety-Critical Policy Evaluation with Accelerated Rare Event Sampling
Mengdi Xu, Peide Huang, Fengpei Li, Jiacheng Zhu, Xuewei Qi, Kentaro Oguchi, Zhiyuan Huang, Henry Lam, Ding Zhao
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022
[paper]

* indicates equal contribution.

2026

Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction
Ruoqu Chen, Feixiang Ruan, Liu Cao, Zihao Wang, Botian Xu, Shiqin Tong, Jiajun Liu, Wanli Xing, Kaifeng Zhang, Mengdi Xu
RSS 2026 Workshop on Dexterous Manipulation (spotlight).
[project]

Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment
Jacky Kwok*, Xilun Zhang*, Mengdi Xu, Yuejiang Liu†, Azalia Mirhoseini†, Chelsea Finn†, Marco Pavone†
In submission, 2026. † equal advising.
[paper] [website]

2025

MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning
Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu*, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhogn Tu, Laixi Shi, Jiacheng Zhu
MMRAgL workshop at ICCV 2025 (oral presentation)
[paper] [website]

2024

Embodied Executable Policy Learning with Language-based Scene Summarization
Jielin Qiu*, Mengdi Xu*, William Han*, Seungwhan Moon, Ding Zhao
2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)
Abridged in ICML 2023 Workshop on Interactive Learning with Implicit Human Feedback (spotlight)
[paper]

Guardians as You Fall: Active Mode Transition for Safe Falling
Yikai Wang, Mengdi Xu, Guanya Shi, Ding Zhao
Preprint, under review
[paper] [website] [code] [video]

2023

Creative Robot Tool Use with Large Language Models
Mengdi Xu*, Peide Huang*, Wenhao Yu*, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, Ding Zhao
Preprint.
Abridged in CoRL 2023 Workshop on Language and Robot Learning: Language as Grounding
[paper] [webpage] [MLD Blog] [TechXplore]

Adaptive Online Replanning with Diffusion Models
Siyuan Zhou, Yilun Du, Shun Zhang, Mengdi Xu, Yikang Shen, Wei Xiao, Dit-Yan Yeung, Chuang Gan
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023
[paper] [webpage]

What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discovery
Peide Huang, Xilun Zhang*, Ziang Cao*, Shiqi Liu*, Mengdi Xu, Wenhao Ding, Jonathan Francis, Bingqing Chen, Ding Zhao
Conference on Robot Learning (CoRL), 2023
Abridged in IROS 2023 Workshop on Causality for Robotics: Answering the Question of Why.
[paper] [webpage]

Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables
Mengdi Xu, Peide Huang, Yaru Niu, Visak Kumar, Jielin Qiu, Chao Fang, Kuan-Hui Lee, Xuewei Qi, Henry Lam, Bo Li, Ding Zhao
The 26th International Conference on Artificial Intelligence and Statistics, (AISTATS), 2023
Abridged in ICML 2022 workshop on Principles of Distribution Shift.
Abridged in ICRA 2022 the Fresh Perspectives on the Future of Autonomous Driving Workshop
[paper] [webpage]

A trajectory is worth three sentences: multimodal transformer for offline reinforcement learning
Yiqi Wang, Mengdi Xu, Laixi Shi, Yuejie Chi
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, (UAI), 2023
[paper] [code]

Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models?
Jielin Qiu, William Han, Jiacheng Zhu, Mengdi Xu, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao
The 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2023
[paper]

2022

Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptation
Peide Huang, Mengdi Xu, Jiacheng Zhu, Laixi Shi, Fei Fang, Ding Zhao,
The 36th Conference on Neural Information Processing Systems, (NeurIPS), 2022
[paper]

Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Training
Peide Huang, Mengdi Xu, Fei Fang, Ding Zhao
International Joint Conference on Artificial Intelligence (IJCAI), 2022
[paper] [bibtex]

Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability
Mengdi Xu*, Zuxin Liu*, Peide Huang*, Wenhao Ding, Zhepeng Cen, Bo Li, Ding Zhao
Preprint, under review
[paper]

2021

Context-Aware Safe Reinforcement Learning for Non-Stationary Environments
Baiming Chen, Zuxin Liu, Jiacheng Zhu, Mengdi Xu, Wenhao Ding, Liang Li, Ding Zhao
IEEE International Conference on Robotics and Automation (ICRA), 2021
[paper] [bibtex]

Delay-Aware Model-Based Reinforcement Learning for Continuous Control
Baiming Chen, Mengdi Xu, Liang Li, Ding Zhao
Neurocomputing, 2021
[paper] [code] [bibtex]

Functional Optimal Transport: Map Estimation and Domain Adaptation for Functional data
Jiacheng Zhu*, Aritra Guha*, Dat Do*, Mengdi Xu, XuanLong Nguyen, Ding Zhao
JMLR
Abridged in AAAI OT-SDM 2022 workshop (spotlight)
[paper] [code] [bibtex]

Delay-Aware Multi-Agent Reinforcement Learning
Baiming Chen, Mengdi Xu, Zuxin Liu, Liang Li, Ding Zhao
Preprint, under review
[paper] [code] [bibtex]

2020

Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes
Mengdi Xu, Wenhao Ding, Jiacheng Zhu, Zuxin Liu, Baiming Chen, Ding Zhao
Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS), 2020
[paper] [video] [code] [bibtex]

CMTS: Conditional Multiple Trajectory Synthesizer for Generating Safety-critical Driving Scenarios
Wenhao Ding, Mengdi Xu, Ding Zhao
International Conference on Robotics and Automation (ICRA), 2020
[paper] [code] [bibtex]

2019

Mosquito staging apparatus for producing PfSPZ malaria vaccines
Mengdi Xu, Shengnan Lyu, Yingtian Xu, Can Kocabalkanli, Brian K. Chirikjian, John S. Chirikjian, Joshua Davis, Jin Seob Kim, Iulian Iordachita, Russell H. Taylor, Gregory S. Chirikjian
IEEE 15th International Conference on Automation Science and Engineering (CASE), 2019
[paper] [bibtex]

Multi-Mosquito Object Detection and 2D Pose Estimation for Automation of PfSPZ Malaria Vaccine Production
Hongtao Wu, Jiteng Mu, Ting Da, Mengdi Xu, Russell H. Taylor, Iulian Iordachita, Gregory S. Chirikjian
IEEE 15th International Conference on Automation Science and Engineering (CASE), 2019
[paper] [bibtex]

2018

Recovering a Rotation Matrix From Three Direction Cosines
Mengdi Xu, Gregory S. Chirikjian
ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC-CIE), 2018
[paper] [bibtex]

Influence of hydrodynamic pressure and vein strength on the super-elasticity of honeybee wings
Jieliang Zhao*, Mengdi Xu* Youjian Liang, Shaoze Yan, Wendong Niu
Journal of insect physiology, 2018
[paper] [bibtex]

Talks

Research Overview March 2024

Awards

Best PhD Dissertation Award 2024, Mechanical Engineering, CMU, 2024
EECS Rising Stars, 2023
Robotics: Science and Systems (RSS) Pioneers, 2023
Rising Stars in Computational and Data Sciences, 2023

Service

Area Chair for ACL ARR
Co-organized The first BEHAVIOR Challenge 2025
Co-organized RSS 2024 Workshop on Lifelong Robot Learning: Generalization, Adaptation, and Deployment with Large Models
Co-organized RSS Pioneers Workshop 2024
Breakout session leader at 3rd Women in Machine Learning Un-Workshop, ICML 2022
Mentor of 2020 CMU Robotics Institute Summer Scholars Program (RISS)
Conference Reviewer: NeurIPS, ICML, ICLR, CoRL, AISTATS, ACL, ICRA, ICCV, ECCV, CVPR, AAAI, L4DC, ICASSP
Journal Reviewer: IJRR, RA-L, T-ITS, T-IV

Teaching

Interactive AI, 2025 Fall

Mentoring

Xilun Zhang (Master, CMU → PhD, Stanford)
Shiqi Liu (Master, CMU → PhD, CMU)
Ziang Cao (Undergrad, UPitt → Master, Stanford)

Website template from Jon Barron.