GeezerVison

GeezerVison is showing lupantech.github.io — Pan Lu

I am a Postdoctoral Scholar at Stanford University. I am affiliated with Stanford NLP Group and Stanford AI Lab, where I am fortunate to be advised by Professor James Zou and Professor Yejin Choi.

I received my Ph.D. in computer science from UCLA, where I was advised by Kai-Wei Chang and Song-Chun Zhu. I was a member of UCLA Natural Language Processing Group (UCLA NLP). Previously, I completed my M.S. in computer science at Tsinghua University, supervised by Jianyong Wang. My research has been recognized with Most Influential ICLR Paper Award (top-15 cited at ICLR 2024), Most Influential NIPS Paper Award (top-15 cited at NeurIPS 2022), KnowledgeNLP 2025 Workshop Best Paper Award, and EMNLP 2024 Best Paper Nomination. I have been fortunate to receive recognition from Amazon PhD Fellowship, Bloomberg Data Science Ph.D. Fellowship (Global 9), Qualcomm Innovation Fellowship (18 winners), and UCLA Dissertation Year Fellowship. I am fortunate to receive a AI for Math Fund Grant ($1M for two years) from the Renaissance Philanthropy Foundation.

My research goal is to develop intelligent machines that can reason and collaborate with humans for the common good. My primary focus lies in machine learning and natural language processing, particularly in machine reasoning, mathematical reasoning, and scientific discovery. My recent research interests include:

LLM Agents and Agentic Systems for complex reasoning [AgentFlow] [LatentMAS] [OctoTools] [TaTToo] [AgentDebug] [W4S]
AI for Science: AI for scientific reasoning and discovery [Eubiota] [iSight] [ScienceQA] [SciBench] [Protein-LLM] [ChemAgent]
AI for Math: advancing mathematical reasoning of AI systems and LLMs across multimodal, knowledge-intensive, and real-world contexts [IneqMath] [MathVista] [MathVerse] [PromptPG] [Inter-GPS] [IconQA] [TheoremQA] [DL4Math] [MATH-AI]
Post-Training and Test-Time Training techniques for foundation models [STIC] [LLaMA-Adapter] [LLaMA-Adapter V2] [SPHINX-X] [TextGrad] [PromptPG]

News

[07/2026] New! Excited to have DeeptraceReward on AI-generated video detection accepted at COLM 2026!
[06/2026] New! Honored to be invited as a lecturer at MLSS 2026 at Columbia University!
[04/2026] New! Excited that LatentMAS was selected for Spotlight Presentation (Top 2.2%) at ICML 2026! 🎉
[04/2026] New! Excited to have two papers accepted at ICML 2026! See you in Seoul, South Korea!
[04/2026] New! Excited that OctoTools was selected for Oral Presentation (Top 3.4%) at ACL 2026!
[03/2026] New! Thrilled to release Eubiota, agentic AI for autonomous discovery in the gut microbiome! 🎉
[03/2026] New! Excited to release Eubiota with the interactive app! 🎉
[02/2026] New! Excited that AgentFlow was selected for Oral Presentation (Top 1.1%) at ICLR 2026! 🎉
[02/2026] New! Honored to be invited to serve as a reviewer for Nature Human Behaviour!
[01/2026] New! Excited to have two papers accepted at ICLR 2026!
[12/2025] New! Excited to be invited to serve as a Senior Area Chair for ACL 2026!
[12/2025] New! Excited that AgentFlow won 🏆 Best Paper Runner-up at the Effective Reasoning Workshop!
[12/2025] New! Excited to co-organize the 5th MATH-AI Workshop at NeurIPS 2025! See you in San Diego!
[11/2025] New! Our paper on latent collaboration in multi-agent systems LatentMAS is available at Preprint.
[10/2025] New! Our trainable agentic framework AgentFlow for planning and tool use is available at Preprint.
[09/2025] New! Honored to be selected as a World's Top 2% Scientists by Elsevier!
[09/2025] New! Excited to have our IneqMath paper accepted at NeurIPS 2025 as a Spotlight 🎉!
[08/2025] New! Excited to win the AI for Math Fund Grant Award ($1M Grant for two years) 🏆!
[07/2025] New! Glad to co-organize the 5th MATH-AI Workshop at NeurIPS 2025!

[06/2025] New! Glad to co-organize the Multimodal Mathematical Reasoning Workshop at CVPR 2025!
[06/2025] New! Our new study on solving inequality proofs with LLMs is available at Preprint.
[05/2025] New! Excited that MathVista has been recognized as the 🏆 Most Influential ICLR Paper (ICLR-24)!
[05/2025] New! Excited that OctoTools received the 🏆 Best Paper Award at the KnowledgeNLP Workshop!
[04/2025] New! Excited to be invited to serve as a Senior Area Chair for EMNLP 2025!
[04/2025] New! Excited to be invited to serve as an Area Chair for NeurIPS 2025!
[03/2025] New! Thrilled to announce that TextGrad is published in Nature!
[02/2025] New! Our OctoTools agentic framework with extensible tools is available at Preprint.
[01/2025] New! Four papers are accepted to ICLR 2025.
[12/2024] New! I am co-organizing the 4th MATH-AI Workshop at NeurIPS 2024. See you in Vancouver!
[09/2024] New! One paper on self-improving vision-language models is accepted to NeurPIS 2024.
[09/2024] New! Excited to have our NeurIPS 2022 paper recognized as Most Influential NIPS Papers!
[09/2024] New! One paper on self-improving vision-language models is accepted to NeurPIS 2024.
[09/2024] New! Three papers are accepted to EMNLP 2024. See you in Miami!
[07/2024] New! One paper on visual math problems is accepted to ECCV 2024.
[06/2024] New! A paper on debugging visual programs is available at Preprint.
[06/2024] New! A paper on multi-image understanding is available at Preprint.
[05/2024] New! A paper on enhancing LVLMs with self-training is available at Preprint.
[05/2024] New! Thrilled to be awarded the Bloomberg Data Science Ph.D. Fellowship! Thanks!
[05/2024] New! One paper on advanced quantitative reasoning is accepted to ACL 2024 (Findings).
[05/2024] New! Two papers on math reasoning and VLMs are accepted at ICML 2024. See you in Vienna!
[04/2024] New! Defended my doctoral dissertation! Thanks to my advisor and committee members!
[03/2024] New! I am co-organizing the AI for Math Workshop at ICML 2024. See you in Vienna!
[03/2024] New! A paper on visual math reasoning with Multi-modal LLMs is available at Preprint.
[02/2024] New! A paper on LLMs for advanced quantitative reasoning is available at Preprint.
[01/2024] New! Two papers on large multimodal models are accepted to ICLR 2024.
[01/2024] New! A paper on model editing for LLMs is available at Preprint.
[01/2024] New! Two papers on large multimodal models are accepted to ICLR 2024.
[01/2024] New! A paper on model editing for LLMs is available at Preprint.
[12/2023] New! I am co-organizing the Tool-Augmented VIsion Workshop at CVPR 2024. See you in Seattle!
[12/2023] New! I am attending NeurIPS 2023 from Dec 10 to Dec 16. See you in New Orleans!
[12/2023] New! Google's Gemini benchmarks our MathVista for evaluating math reasoning in visual contexts!
[11/2023] New! Honored to be covered by UCLA CS for winning Qualcomm Innovation Fellowship. Thanks!
[10/2023] New! The 112-page study on GPT-4V, Bard, and others on visual math reasoning is available here.
[10/2023] New! Honored to serve as PC Chair and co-organize SoCal NLP 2023. See you in LA!
[10/2023] New! One paper on mathematical reasoning is accepted to EMNLP 2023.
[10/2023] New! One paper on mathematical reasoning in visual contexts (MathVista) is submitted to Preprint.
[09/2023] New! One paper on tool-augmented LLMs is accepted to NeurIPS 2023.
[07/2023] New! One paper on a scientific reasoning benchmark (SciBench) is submitted to Preprint.
[07/2023] New! I am co-organizing the 3rd MATH-AI Workshop at NeurIPS 2023. See you in New Orleans!
[06/2023] New! Excited to receive the UCLA Dissertation Year Fellowship.
[05/2023] New! Honored to deliver a guest lecture for UCLA CS 263: Natural Language Processing. [Slides]
[05/2023] New! One paper on theorem-driven math question answering (TheoremQA) is available at Preprint.
[05/2023] New! Honored to deliver a invited talk on tool-augmented LLMs at Google Brain. [Slides]
[05/2023] New! Delighted to join prestigious LightingAI event as invited speaker on Discord.
[05/2023] New! A paper on multimodal procedural planning is available at Preprint.
[05/2023] New! One survey paper on deep learning for mathematical reasoning is accepted to ACL 2023.
[04/2023] New! LLaMA-Adapter-V2, a parameter-efficient visual instruction model, is available at Preprint.
[04/2023] New! One tutorial proposal on mathematical reasoning is accepted to IJCAI 2023.
[04/2023] New! One paper on tool augmented LLMs (Chameleon) is available at Preprint.
[04/2023] New! Two papers are accepted to CVPR 2023 O-DRUM Workshop.
[03/2023] New! One paper on fine-tuning LLaMA in one hour (LLaMA-Adapter) is available at Preprint.
[01/2023] New! One paper on in-context learning for math reasoning (PromptPG) is accepted to ICLR 2023.
[12/2022] New! A survey paper on deep learning for mathematical reasoning is available at Preprint.
[12/2022] New! One paper is accepted to AAAI'23 KnowledgeNLP Workshop as an Oral Presentation.
[12/2022] New! I am excited to join Microsoft Research as a research intern!
[10/2022] New! Happy to receive the NeurIPS 2022 Scholar Award.
[10/2022] New! Two papers on mathematical reasoning are accepted to EMNLP 2022.
[09/2022] New! One paper on prompt learning for math reasoning (PromptPG) is submitted to Preprint.
[09/2022] New! One paper on chain-of-thought reasoning for ScienceQA is accepted to NeurIPS 2022.
[07/2022] New! I am co-organizing the 2nd MATH-AI Workshop at NeurIPS 2022. See you in New Orleans!
[07/2022] New! One paper on socially intelligent agents is accepted to SIGDIAL 2022.
[04/2022] Excited to be listed as a Highlighted Reviewer for ICLR 2022.
[03/2022] I am excited to join Allen Institute for AI (AI2) as a research intern!
[03/2022] One paper on character animation sampling is submitted to Preprint.
[12/2021] Two papers are accepted to AAAI 2022.
[10/2021] One paper on visual question answering for icon images (IconQA) is accepted to NeurIPS 2021.
[07/2021] I am co-organizing the MATHAI4ED Workshop at NeurIPS 2021. Welcome to participate!
[07/2021] Our workshop proposal for Math AI for Education (MATHAI4ED) is accepted to NeurIPS 2021.
[05/2021] One paper on interpretable geometry problem solving is accepted to ACL 2021 as an Oral Presentation.
[05/2021] One paper on social relation inference in dialogues is accepted to ACL 2021 as an Oral Presentation.
[03/2021] One paper on socially intelligent agents is submitted to Preprint.

Upcoming Travel

November 16 - November 17, 2026: MLCB 2026, New York City
December 10 - December 13, 2026: NeurIPS 2026, Atlanta, Georgia
April 26 - April 30, 2027: ICLR 2027, San Francisco, CA

Selected Publications

All publications can be found on my Google Scholar page.

Eubiota: Modular Agentic AI for Autonomous Discovery in the Gut Microbiome
Pan Lu*, Yifan Gao*, William G. Peng, Haoxiang Zhang, Kunlun Zhu, Elektra K. Robinson, Qixin Xu, Masakazu Kotaka, Harrison G. Zhang, Bingxuan Li, Anthony L. Shiver, Yejin Choi, Kerwyn Casey Huang, Justin Sonnenburg†, James Zou†
bioRxiv:2602.27708412 [Project] [App] [Code] [Model] [Data] [Twitter] [BibTex]
(*Co-first authors) (†Corresponding authors)

In-the-Flow Agentic System Optimization for Effective Planning and Tool Use
Zhuofeng Li*, Haoxiang Zhang*, Seungju Han, Sheng Liu, Jianwen Xie, Yu Zhang, Yejin Choi, James Zou†, Pan Lu*†
ICLR 2026 [Project] [Paper] [Code] [Model] [Demo] [YouTube] [Twitter] [Slack] [BibTex]

(*Equal Contribution) (†Co-senior authors)
🏆 ICLR 2026 Oral (Top 1.1%)
🏆 Best Paper Runner-up, NeurIPS 2025 ER Workshop
#1 Space of the week on Hugging Face
#2 Paper of the day on Hugging Face

Latent Collaboration in Multi-Agent Systems
Jiaru Zou*, Xiyuan Yang*, Ruizhong Qiu*, Gaotang Li*, Katherine Tieu*, Pan Lu*, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He†, James Zou†, Mengdi Wang†, Ling Yang†
ICML 2026 [Paper] [Code] [BibTex]

(*Core Contribution) (†Co-senior authors)
🏆 ICML 2026 Spotlight (Top 2.2%)
#1 Paper of the day on Hugging Face

Proteo-R1: Thinking Foundation Models for De Novo Protein Binder Design
Fang Wu, Li Erran Li, Weihao Xuan, Heli Qi, Zeqi Zhou, Hanqun Cao, Heng-Jui Chang, Haokai Zhao, Jian Ma, Zijian Carl Ma, Yu-Chi Cheng, Xiangru Tang, Zehong Wang, Kuan Pang, Hanchen Wang, Kejun Ying, Pan Lu, Chiho Im, Seungju Han, Peng Xia, Yinxi Li, Guanlue Li, Tinson Xu, Deyao Zhu, Pheng-Ann Heng, Naoto Yokoya, Masashi Sugiyama, Jure Leskovec, Yejin Choi
ICML 2026 [BibTex]

Recursive Multi-Agent Systems
Xiyuan Yang, Jiaru Zou, Rui Pan, Ruizhong Qiu, Pan Lu, Shizhe Diao, Jindong Jiang, Hanghang Tong, Tong Zhang, Markus J. Buehler, Jingrui He, James Zou
arXiv:2604.25917 [Project] [arXiv] [HF Paper] [Code] [Model] [Twitter] [BibTex]

#1 Paper of the day on Hugging Face

OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning
Pan Lu*, Bowen Chen*, Sheng Liu*, Rahul Thapa, Joseph Boen, James Zou
ACL 2026 [Project] [Paper] [Code] [Package] [Demo] [YouTube] [Twitter] [Slack] [BibTex]

(*Equal Contribution)
🏆 ACL 2026 Oral (Top 3.4%)
🏆 Best Paper Award, NAACL 2025 KnowledgeNLP Workshop

iSight: Towards expert-AI co-assessment for improved immunohistochemistry staining interpretation
Jacob S. Leiby*, Jialu Yao*, Pan Lu*, George Hu, Anna Davidian, Shunsuke Koga, Olivia Leung, Pravin Patel, Isabella Tondi Resta, Rebecca Rojansky, Derek Sung, Eric Yang, Paul J. Zhang, Emma Lundberg, Dokyoon Kim, Serena Yeung-Levy, James Zou, Thomas Montine, Jeffrey Nirschl, Zhi Huang
arXiv:2602.04063 [Paper] [Data] [Code] [BibTex]
*Equal Contribution

TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning
Jiaru Zou, Soumya Roy, Vinay Kumar Verma, Ziyi Wang, David Wipf, Pan Lu, Sumit Negi, James Zou, Jingrui He
ICLR 2026 [Paper] [BibTex]

Adaptation of Agentic AI
Pengcheng Jiang*, Jiacheng Lin*, Zhiyi Shi*, Zifeng Wang, Luxi He, Yichen Wu, Ming Zhong, Peiyang Song, Qizheng Zhang, Heng Wang, Xueqiang Xu, Hanwen Xu, Pengrui Han, Dylan Zhang, Jiashuo Sun, Chaoqi Yang, Kun Qian, Tian Wang, Changran Hu, Manling Li, Quanzheng Li, Hao Peng, Sheng Wang, Jingbo Shang, Chao Zhang, Jiaxuan You, Liyuan Liu, Pan Lu, Yu Zhang, Heng Ji, Yejin Choi, Dawn Song, Jimeng Sun, Jiawei Han†
arXiv:2512.16301 [Paper] [Code] [Coverage] [BibTex]

(*Equal Contribution) (†Corresponding author)
#2 Paper of the day on Hugging Face

Where LLM Agents Fail and How They can Learn From Failures
Kunlun Zhu*, Zijia Liu*, Bingxuan Li*, Muxin Tian*, Yingxuan Yang, Jiaxun Zhang, Pengrui Han, Qipeng Xie, Fuyang Cui, Weijia Zhang, Xiaoteng Ma, Xiaodong Yu, Gowtham Ramesh, Jialian Wu, Zicheng Liu, Pan Lu†, James Zou†, Jiaxuan You†
arXiv:2509.25370 [Paper] [Code] [BibTex]
(*Equal Contribution) (†Co-senior authors)

Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs
Xingyu Fu, Siyi Liu, Yinuo Xu, Pan Lu, Guangqiuse Hu, Tianbo Yang, Taran Anantasagar, Christopher Shen, Yikai Mao, Yuanzhe Liu, Keyush Shah, Chung Un Lee, Yejin Choi, James Zou, Dan Roth, Chris Callison-Burch
COLM 2026 [Project] [Paper] [Code] [Data] [BibTex]

Fractional Reasoning via Latent Steering Vectors Improves Inference Time Compute
Sheng Liu, Tianlang Chen, Pan Lu, Haotian Ye, Yizheng Chen, Lei Xing, James Zou
2506.15882 [Project] [Paper] [Code] [BibTex]

Solving Inequality Proofs with Large Language Models
Pan Lu*, Jiayi Sheng*, Luna Lyu*, Jikai Jin, Tony Xia, Alex Gu, James Zou
NeurIPS 2025 [Project] [Paper] [Code] [Data] [Submission] [Twitter] [BibTex]
(*Equal Contribution)
🏆 NeurIPS 2025 Spotlight (Top 3.0%)

Advancing AI Research Assistants with Expert-Involved Learning
Tianyu Liu, Simeng Han, Hanchen Wang, Xiao Luo, Pan Lu, Biqing Zhu, Yuge Wang, Keyi Li, Jiapeng Chen, Rihao Qu, Yufeng Liu, Xinyue Cui, Aviv Yaish, Yuhang Chen, Minsheng Hao, Chuhan Li, Kexing Li, Arman Cohan, Hua Xu, Mark Gerstein, James Zou, Hongyu Zhao
arXiv:2505.04638 [Paper] [BibTex]

Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executors
Fan Nie, Lan Feng, Haotian Ye, Weixin Liang, Pan Lu, Huaxiu Yao, Alexandre Alahi, James Zou
COLM 2025 [Paper] [Code] [BibTex]

Optimizing generative AI by backpropagating language model feedback
Mert Yuksekgonul*, Federico Bianchi*, Joseph Boen*, Sheng Liu*, Pan Lu*, Zhi Huang*, Carlos Guestrin, James Zou
Nature 639, 609–616 (2025) [Project] [Paper] [Code] [YouTube] [Documentation] [BibTex]

(*Equal Contribution)

Protein Large Language Models: A Comprehensive Survey
Yijia Xiao, Wanjia Zhao, Junkai Zhang, Yiqiao Jin, Han Zhang, Zhicheng Ren, Renliang Sun, Haixin Wang, Guancheng Wan, Pan Lu, Xiao Luo, Yu Zhang, James Zou, Yizhou Sun, Wei Wang
EMNLP 2025 (Findings) [Paper] [PDF] [Tutorial] [Coverage] [BibTex]

VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning
Xueqing Wu, Yuheng Ding, Bingxuan Li, Pan Lu, Da Yin, Kai-Wei Chang, Nanyun Peng
CVPR 2025 [Project] [Paper] [PDF] [Code] [Data] [BibTex]

ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning
Xiangru Tang, Tianyu Hu, Muyang Ye, Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, Arman Cohan, Mark Gerstein
ICLR 2025 [Paper] [PDF] [Code] [News] [BibTex]

MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models
Wenbo Hu, Jia-Chen Gu, Zi-Yi Dou, Mohsen Fayyaz, Pan Lu, Kai-Wei Chang, Nanyun Peng
ICLR 2025 [Project] [Paper] [PDF] [Code] [HF Dataset] [BibTex]

MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines
Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Guanglu Song, Peng Gao, Yu Liu, Chunyuan Li, Hongsheng Li
ICLR 2025 [Project] [Paper] [PDF] [Hugging Face] [Code] [Data] [BibTex]

MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding
Fei Wang*, Xingyu Fu*, James Y. Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, Tianyi Lorena Yan, Wenjie Jacky Mo, Hsiang-Hui Liu, Pan Lu, Chunyuan Li, Chaowei Xiao, Kai-Wei Chang, Dan Roth, Sheng Zhang, Hoifung Poon, Muhao Chen
ICLR 2025 [Project] [Paper] [PDF] [Hugging Face] [Code] [Data] [Twitter] [BibTex]
(*Equal Contribution)

Enhancing Large Vision Language Models with Self-Training on Image Comprehension
Yihe Deng*, Pan Lu*, Fan Yin, Ziniu Hu, Sheng Shen, Quanquan Gu, James Zou, Kai-Wei Chang, Wei Wang
NeurIPS 2024 [Project] [Paper] [PDF] [Hugging Face] [Code] [Model] [Data] [BibTex]
(*Equal Contribution)

Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue
Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, Nanyun Peng
EMNLP 2024 [Paper] [PDF] [Code] [Twitter] [BibTex]
🏆 Best Paper Nomination, EMNLP 2024

VDebugger: Harnessing Execution Feedback for Debugging Visual Programs
Xueqing Wu, Zongyu Lin, Songyan Zhao, Te-Lin Wu, Pan Lu, Nanyun Peng, Kai-Wei Chang
EMNLP 2024 (Findings) [Project] [Paper] [PDF] [Code] [Model] [Data] [Twitter] [BibTex]

Multimodal Procedural Planning via Dual Text-Image Prompting
Yujie Lu, Pan Lu, Zhiyu Chen, Wanrong Zhu, Xin Eric Wang, William Yang Wang
EMNLP 2024 (Findings) [Paper] [PDF] [Code] [Twitter] [Coverage] [BibTex]

MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, Hongsheng Li
ECCV 2024 [Project] [Paper] [PDF] [Code] [Data] [Visualization] [Coverage] [Daily Papers] [BibTex]

Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data
Xiao Liu, Zirui Wu, Xueqing Wu, Pan Lu, Kai-Wei Chang, Yansong Feng
ACL 2024 (Findings) [Project] [Paper] [PDF] [Code] [Data] [Twitter] [BibTex]

SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
Xiaoxuan Wang*, Ziniu Hu*, Pan Lu*, Yanqiao Zhu*, Jieyu Zhang, Satyen Subramaniam, Arjun R. Loomba, Shichang Zhang, Yizhou Sun, Wei Wang
ICML 2024 [Paper] [PDF] [Code] [Twitter] [BibTex]

(*Equal Contribution)
Nature News Feature (15 November 2023)

SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
Peng Gao, Renrui Zhang, Chris Liu, Longtian Qiu, Siyuan Huang, Weifeng Lin, Shitian Zhao, Shijie Geng, Ziyi Lin, Peng Jin, Kaipeng Zhang, Wenqi Shao, Chao Xu, Conghui He, Junjun He, Hao Shao, Pan Lu, Hongsheng Li, Yu Qiao
ICML 2024 [Paper] [PDF] [Code] [Doc] [Hugging Face] [Twitter] [Coverage] [BibTex]

MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, Jianfeng Gao
ICLR 2024 [Project] [Paper] [PDF] [Code] [Dataset] [Leaderboard] [Visualize] [Coverage] [BibTex]

🏆 Most Influential ICLR Papers (Top-15 cited paper at ICLR-24)
🏆 Oral Presentation (Top 1.2%) (85 in 7304 submissions)

LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, Yu Qiao
ICLR 2024 [Paper] [PDF] [Code] [Twitter] [Coverage] [BibTex]

LightningAI Blog Feature (14 April 2023)

Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Jianfeng Gao
NeurIPS 2023 [Project] [Paper] [PDF] [Code] [Twitter] [Coverage] [BibTex]

🏆 Best Weekly AI Paper (by AlphaSignal, 1st in 1682, 0.06%)
Awesome NeurIPS 2023 Papers (40 in 3584, 0.01%)
NeurIPS 2023 Top 10 Multimodal ML Papers

KokoMind: Can LLMs Understand Social Interactions?
Weiyan Shi*, Liang Qiu*, Dehong Xu, Pengwei Sui, Pan Lu, Zhou Yu
[Project] [Code] [Twitter] [Twitter] [BibTex]

(*Equal Contribution)

TheoremQA: A Theorem-driven Question Answering Dataset
Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, Tony Xia
EMNLP 2023 [Paper] [PDF] [Code] [Twitter] [BibTex]

LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, Yu Qiao
arXiv:2304.15010 [Paper] [PDF] [Code] [Gradio] [Gradio-Multimodal] [Twitter] [YouTube] [BibTex]

A Survey of Deep Learning for Mathematical Reasoning
Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, Kai-Wei Chang
ACL 2023 [Paper] [PDF] [Code] [Poster] [Twitter] [Coverage] [BibTex]

🏆 Most Influential ArXiv (Artificial Intelligence) Papers (Top-25 cited paper at arXiv-22)

Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, Ashwin Kalyan
ICLR 2023 [Paper] [PDF] [Project] [Data] [Code] [Explore] [Leaderboard] [Twitter] [BibTex]

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, Ashwin Kalyan
NeurIPS 2022 [Paper] [PDF] [Project] [Data] [Huggingface] [Code] [Explore] [Leaderboard] [Twitter] [BibTex]

🏆 Most Influential NIPS Papers (Top-15 cited paper at NeurIPS-22)

LILA: A Unified Benchmark for Mathematical Reasoning
Swaroop Mishra*, Matthew Finlayson*, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, Ashwin K. Kalyan
EMNLP 2022 [Paper] [PDF] [Project] [Data] [Code] [Huggingface] [BibTex]
(*Equal Contribution)

UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression
Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen and Xiaodan Liang
EMNLP 2022 [Paper] [PDF] [Code] [BibTex]

Towards Socially Intelligent Agents with Mental State Transition and Human Utility
Liang Qiu*, Yizhou Zhao*, Yuan Liang, Pan Lu, Weiyan Shi, Zhou Yu, Song-Chun Zhu
SIGDIAL 2022 [Paper] [PDF] [BibTex]
(*Equal Contribution)

Learning from the Tangram to Solve Mini Visual Tasks
Yizhou Zhao, Liang Qiu, Pan Lu, Feng Shi, Tian Han, Song-Chun Zhu
AAAI 2022 [Paper] [PDF] [Code] [BibTex]
Oral Presentation

ValueNet: A New Dataset for Human Value Driven Dialogue System
Liang Qiu, Yizhou Zhao, Jinchao Li, Pan Lu, Baolin Peng, Jianfeng Gao, Song-Chun Zhu
AAAI 2022 [Paper] [PDF] [Project] [BibTex]

IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, Song-Chun Zhu
NeurIPS 2021 [Paper] [PDF] [Project] [Code] [BibTex]
Datasets and Benchmarks Track

Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning
Pan Lu*, Ran Gong*, Shibiao Jiang*, Liang Qiu, Siyuan Huang, Xiaodan Liang, Song-Chun Zhu
ACL 2021 [Paper] [PDF] [Project] [Code] [BibTex]

Oral Presentation (*Equal Contribution)

SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues
Liang Qiu, Yuan Liang, Yizhou Zhao, Pan Lu, Baolin Peng, Zhou Yu, Ying Nian Wu, Song-Chun Zhu
ACL 2021 [Paper] [PDF] [BibTex]
Oral Presentation

Learning Long- and Short-Term User Literal-Preference with Multimodal Hierarchical Transformer Network for Personalized Image Caption
Wei Zhang, Yue Ying, Pan Lu, Hongyuan Zha
AAAI 2020 [Paper] [PDF] [BibTex]

Knowledge Aware Semantic Concept Expansion for Image-Text Matching
Botian Shi, Lei Ji, Pan Lu, Nan Duan
IJCAI 2019 [Paper] [BibTex]
Oral Presentation

Dynamic Fusion with Intra- and Inter-modality Attention Flow for Visual Question Answering
Peng Gao, Zhengkai Jiang, Haoxuan You, Pan Lu, Steven CH Hoi, Xiaogang Wang, Hongsheng Li
CVPR 2019 [Paper] [Code] [BibTex]
Oral Presentation

Knowledge-Aware Deep Dual Networks for Text-Based Mortality Prediction
Ning Liu, Pan Lu, Wei Zhang, Jianyong Wang
ICDE 2019 [Paper] [BibTex]

Question-Guided Hybrid Convolution for Visual Question Answering
Peng Gao, Hongsheng Li, Shuang Li, Pan Lu, Yikang Li, Steven Hoi, Xiaogang Wang
ECCV 2018 [Paper] [BibTex]

R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering
Pan Lu, Lei Ji, Wei Zhang, Nan Duan, Ming Zhou, Jianyong Wang
SIGKDD 2018 [Paper] [Data] [Video] [BibTex]
Oral Presentation

Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
Pan Lu, Hongsheng Li, Wei Zhang, Jianyong Wang, Xiaogang Wang
AAAI 2018 [Paper] [Code] [BibTex]
Oral Presentation

Teaching

Guest Lecturers

Lecturer, Machine Learning Summer School (MLSS) 2026 [Website]
Columbia University, Summer 2026
Guest Lecturer, CSE 244C: Deep Learning for Advanced Computer Vision [Website]
Graduate, UC Santa Cruz, Spring 2025
Guest Lecturer, LLMs and Mathematics Seminar [Website]
Graduate, Rutgers University, Spring 2025
Guest Lecturer, CSCE 638: Natural Language Processing: Foundation and Techniques [Website]
Graduate, TAMU, Spring 2025
Guest Lecturer, CS 263: Natural Language Processing [Slides]
Graduate, UCLA, Spring 2023
Guest Lecturer, ECE C247: Neural Networks and Deep Learning [Slides]
Graduate, UCLA, Winter 2022
Guest Lecturer, Data Mining: Theory and Algorithms
Graduate, Tsinghua University, Fall 2017

Teaching Assistants

Teaching Associate, ECE C247: Neural Networks and Deep Learning
Graduate, UCLA, Winter 2023
Teaching Assistant, CS 249: Data Science Fundamentals
Graduate, UCLA, Fall 2022
Teaching Assistant, ECE C247: Neural Networks and Deep Learning
Graduate, UCLA, Winter 2022
Teaching Assistant, CS 111: Operating System Principles
Undergraduate, UCLA, Fall 2021
Teaching Assistant, C Programming Language
Undergraduate, Tsinghua University, Spring 2016

Mentorship

I am fortunate to work with a group of brilliant students and collaborators.

Students and Collaborators

Lai Jiang: UIUC CS M.S. (PhD Application for Fall 2027)
Edward Sun: UCLA Undergrad (PhD Application for Fall 2027)
Guowei Xu: Visiting Undergrad (PhD Application for Fall 2027)
Qixin Xu: Visiting Undergrad (PhD Application for Fall 2027)
Haoxiang Zhang: UCSD CS M.S. (PhD Application for Fall 2027)
Jusheng Zhang: Visiting Researcher (PhD Application for Fall 2027)

Alumni

William P. Gao: Stanford CS/Math Undergrad (Class of 2028)
Sean Wu: Pepperdine Undergrad → Stanford CS Ph.D.
Yuru Jiang: Visiting Undergrad
Yiwei Yang: UW CS Ph.D.
Rubin Zou: UIUC CS M.S. → Stanford CS Ph.D.
Zhuofeng Li: Visiting Undergrad → TAMU CS Ph.D.
Jiayi Sheng: Visiting Undergrad → CMU CS M.S.
Jinji Kai: Stanford ICME Ph.D.
Luna Lyu: Stanford ICME M.S. → Stanford Ph.D.
Bingxuan Li: UCLA CS M.S. → UIUC CS Ph.D.
Kunlun Zhu: UIUC CS M.S. → UIUC CS Ph.D.
Songyan (Silas) Zhao: UCLA CS M.S. → Newsbreak
Xueqing Wu: UCLA CS Ph.D.
Wenbo (Gordon) Hu: UCLA CS M.S. → UCLA CS Ph.D.
Bryan Zhou: UCLA CS Undergrad → UCLA CS Ph.D.
Tony Xia: UCLA CS Undergrad → Stanford CS M.S. → Amazon
Shibiao Jiang: Visiting Undergrad
Johnson Zhou: UCLA CS Undergrad → Radix Trading

Professional Service

Conferences

Senior Program Chair, New England NLP Meeting (NENLP) 2025, New Haven, 2025.04
Program Chair, The 4th Southern California Natural Language Symposium (SoCal NLP), Los Angeles, 2023.11
Senior Area Chair: ACL 2027, ACL 2026, EMNLP 2026, EMNLP 2025
Area Chair (2026): NeurIPS 2026, ICML 2026, ICLR 2026
Area Chair (2025): NeurIPS 2025, ICLR 2025, ACL 2025

Workshops and Tutorials

Co-organizer, Workshop on MATH-AI: The 6th Workshop on Mathematical Reasoning and AI at NeurIPS 2026
Co-organizer, Workshop on MATH-AI: The 5th Workshop on Mathematical Reasoning and AI at NeurIPS 2025
Co-organizer, Tutorial on Multimodal Mathematical Reasoning at CVPR 2025
Co-organizer, Workshop on MATH-AI: The 4th Workshop on Mathematical Reasoning and AI at NeurIPS 2024
Co-organizer, Workshop on AI for Math at ICML 2024
Co-organizer, Workshop on Tool-Augmented VIsion (TAVI) at CVPR 2024
Co-organizer, Workshop on MATH-AI: The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS 2023
Co-organizer, Tutorial on Deep Learning in Mathematical Reasoning: Recent Advances and Beyond at IJCAI 2023
Co-organizer, Workshop on MATH-AI: Toward Human-Level Mathematical Reasoning at NeurIPS 2022
Co-organizer, Workshop on Math AI for Education: Bridging the Gap Between Research and Smart Education at NeurIPS 2021

Program Committee Member

2024: NeurIPS, ICLR, ICML, CVPR, COLM, ARR, SIGKDD
2023: NeurIPS, ICLR, ACL, EMNLP, AAAI
2022: NeurIPS, ICLR, CVPR, AAAI, COLING
2021: NeurIPS, ICLR, CVPR, ICCV
2020 and before: NeurIPS, AAAI, SIGKDD, ICDM, PAKDD

Journal Reviewer

Grant Reviewer

AI For Math Fund, Renaissance Philanthropy Foundation

Organizations

Chair, IEEE Student Branch at Tsinghua University, Beijing, 2015.10 - 2016.10

Selected Awards

Grants

💰 Stanford HAI Seed Research Grants, 2026 [Link]
💰 AI for Math Fund Grant ($1M for two years), 2025 [Link]

Fellowships

🧑‍🎓 Thinking About Thinking Fellowship, 2026 [Link]
🧑‍🎓 Bloomberg Data Science Ph.D. Fellowship, 2023-2024 [Bloomberg] [UCLA Engineering] [UCLA CS]
🧑‍🎓 Qualcomm Innovation Fellowship, 2023 [Qualcomm] [UCLA CS]
🧑‍🎓 UCLA Dissertation Year Fellowship, 2023
🧑‍🎓 Amazon PhD Fellowship, 2023 [UCLA/Amazon]

Paper Awards

🏆 Best Poster Award, ICML 2026 AI for Science Workshop, 2026 [Website]
🏆 Most Influential ICLR Papers (ICLR 2024, #11 cited), 2025-09 [List]
🏆 Most Influential NIPS Papers (NeurIPS 2022, #12 cited), 2025-09 [List]
🏆 Best Paper Award Nomination, NeurIPS 2025 Effective Reasoning Workshop, 2025 [Link]
🏆 Best Paper Award Nomination, NeurIPS 2025 MATH-AI Workshop, 2025
🏆 Best Paper Award, NAACL 2025 KnowledgeNLP Workshop, 2024 [Link]
🏆 Best Paper Honorable Mention Award, EMNLP 2024, 2024

Recognition

NeurIPS 2026 Top AC Award, 2026
Rising Stars in AI Symposium 2026 (KAUST), 2026 (Selected) [Link]
World's Top 2% Scientists, Elsevier, 2025 [Link]
EMNLP 2024 Diversity and Inclusion Award, 2023
NeurIPS 2023 Scholar Award, 2023
NeurIPS 2022 Scholar Award, 2022
NeurIPS 2022 Top Reviewer, 2022
ICLR 2022 Highlighted Reviewer, 2022
ACL-IJCNLP 2021 Diversity & Inclusion Award, 2021
Outstanding Master Thesis Award, Tsinghua University, 2018
Outstanding Graduate Award of Computer Science, Tsinghua University, 2018
"Stars of Tomorrow" Excellent Intern Award, Microsoft Research, 2018
GuangHua Scholarship Award, Tsinghua University, 2016
Outstanding Undergraduate, Beijing, 2015
Champion of National College Student Innovation Conference, 2014
National Scholarship (for top 2% students), 2014
First Prize of America Mathematical Contest In Modeling (MCM), 2014
Xu Teli President Scholarship (the highest honor for top 2 undergraduates), 2013

Contact

350 Jane Stanford Way
Stanford, CA 94305
lupantech [at] gmail [dot] com
[Google Scholar] | [Semantic Scholar] | [GitHub] | [LinkedIn]