GeezerVison

Yejin Choi The Dieter Schwarz Foundation Professor & Senior Fellow MacArthur Fellow Office: Gates 362 Phone: 650.725.4537 email: [email protected]

Computer Science & HAI     Stanford University     353 Jane Stanford Way     Stanford, CA 94305

News:

- Briefing at the United Nations Security Council
- Named among TIME100 AI 2025
- Named among the AI2050 Senior Fellows in 2024
- Named among TIME100 AI 2023
- Panel at Davos: The Dawn of Artificial General Intelligence
- Podcast "Unconfuse Me" with Bill Gates: Full episode here (audio-only) and Youtube highlights here (videos)
- A TED talk: "Why AI is Incredibly Smart --- and Shockingly Stupid"
- MacArthur Fellow (class of 2022); 2 min YouTube reel →here
- Keynote at ACL: "2082: An ACL Odyssey: The Dark Matter of Language and Intelligence" along with a fireside chat on "The Trajectory of ACL and the Next 60 years" and a pre-recorded talk →here
- An invited article, "The Curious Case of Commonsense Intelligence" for the Daedalus's special issue on AI & Society
- A podcast interview with the Gradient on commonsense and morality
- Featured by New Yorker: "Can Computers Learn Common Sense?"
- The TWIML AI Podcast with Sam Charrington on "Why is language the best medium for reasoning?"
- An interview by Dhruv Batra on Humans of AI: Stories, Not Stats
- Featured by NY Times on Delphi: "Can a Machine Learn Morality?"
- Featured by Quanta Magazine --- 🤖"Common Sense Comes Closer to Computers"🤖
- Our UW Sounding Board team is the winner of the Alexa Prize!

Awards:

Best/Outstanding Paper Awards:
     - Best Paper Honorable Mention at CVPR 2026
     - Best Paper at NeurIPS 2025
     - Best Paper at EMNLP 2025
     - Best Demo Paper at ACL 2025
     - Outstanding Paper at ACL 2025
     - Best Paper at ACL 2023
     - Outstanding Paper at ACL 2023
     - Outstanding Paper at EMNLP 2023
     - Best Paper at NAACL 2022
     - Outstanding Paper at ICML 2022
     - Outstanding Paper at NeurIPS 2021
     - Outstanding Paper at AAAI 2020
     - Marr Prize (Best Paper) at ICCV 2013
Test of Time:
     - Longuet Higgins Prize at CVPR 2021
     - Test of Time Award at ACL 2021
Finalists:
     - Best Paper Nomination at ACL 2019
     - RoboCup Best Paper Finalist at IROS 2019
Other Paper Recognition:
     - Compute Transparency Champion at CVPR 2026

Recognition:

- TIME100 AI (2025)
- AI2050 Senior Fellow (2024)
- TIME100 AI (2023)
- Dieter Schwarz Foundation Professorship (2024 - current)
- Distinguished Research Fellow at the Institute for Ethics in AI at Oxford (2023)
- Wissner-Slivka Chair (2023 - 2024)
- MacArthur Fellow (2022)
- ACL Fellow (2022)
- Brett Helsel Career Development Professorship (2020 - 2023)
- Borg Early Career Award (BECA) (2018)
- IEEE AI's 10 to Watch (2016)

Research Interests 🔥:

• AI for Science: Molecular foundation models at all scales and modalities, protein reasoning models, crystal reasoning models, molecular reasoning and retrosynthesis
• Pluralistic Alignment & AI for humanity: Data and algorithms for pluralistic norms and values, deliberate alignment processes, civic discourse
• Alternative Training and Inference Algorithms: Test-time training, reinforced pre-training, alternative tokenization, cognitive architectures

Recent Talks (2020 - 2026):

Jun 2026	Keynote at the Stanford AIMI Symposium
Jun 2026	Talk at the Conference on Ethics and Agentic AI at Stanford
Apr 2026	Distinguished Lecture at University of Toronto
Jan 2026	Panel at World Economic Forum in Davos: Next Phase of Intelligence
Dec 2025	Keynote at NeurIPS: The Art of (Artificial) Reasoning
Oct 2025	Panel at APEC: Future Tech Forum Series: AI
Oct 2025	Talk at Seoul National University
Oct 2025	Keynote at Global AI Frontiers Symposium: Democratizing Generative AI by Transcending Scaling Laws
Oct 2025	Talk at COLM RAM2 Workshop: The Art of Artificial Reasoning for Small Language Models
Sep 2025	Briefing at the United Nations Security Council →here
Sep 2025	Hans J. Berliner Lecture in Artificial Intelligence at Carnegie Mellon University: Bending Scaling Laws with Brighter Algorithms
Aug 2025	Talk at CogSci: The Role of AI in Theories of the Future
Jul 2025	Talk at ICML Workshop: Multi-Agent Systems in the Era of Foundation Models: Opportunities, Challenges and Futures
Jul 2025	Talk at ICML Workshop: Methods and Opportunities at Small Scale (MOSS)
Jun 2025	Talk at KAIST AI
May 2025	Talk at ATXSummit: Mission Impossible – Democratising Generative AI by Transcending Scaling Laws
Apr 2025	AI Lab Distinguished Lecture at Princeton University
Mar 2025	Talk at GTC: Bending Scaling Laws with Brighter Algorithms
Feb 2025	Talk at Toronto SRI: The enigma of LLMs: On creativity, pluralism, and paradoxes →here
Oct 2024	Talk at the Simons Institute Workshop: Alignment, Trust, Watermarking, and Copyright Issues in LLMs --- due to a scheduling challenge, Taylor Sorensen will present instead
Sep 2024	Talk at the Simons Institute Workshop: Special Year on Large Language Models and Transformers, Part 1 Boot Camp
Jun 2024	Keynote at the DataBricks' Data+AI Summit
May 2024	Keynote at MLSys →here
May 2024	Keynote at the ICLR Workshop on How Far Are We From AGI?
Mar 2024	Keynote at the Weinberg Cognitive Science Symposium
Mar 2024	Colloquium at Harvard Psychology
Dec 2023	Organizing a NeurIPS Workshop -- 🐌 AI Meets Moral Philosophy and Moral Psychology 🦋
Dec 2023	Talk at the NeurIPS Workshop -- Socially Responsible Language Modelling Research (SoLaR)
Nov 2023	Talk at UC Irvine
Nov 2023	Talk at UCSD
Nov 2023	Talk at MIT EECS
Nov 2023	Colloquium at MIT Brain and Cognitive Sciences (BCS)
Nov 2023	Distinguished Lecture at UBC
Oct 2023	Talk at the MacArthur Fellows Forum
Oct 2023	Distinguished Lecture at TTIC
Oct 2023	Talk at the Ohio State University
Sep 2023	Talk at the Department of Philosophy at NYU
Sep 2023	Talk at Cornell Tech
Sep 2023	Talk at MSR @ NYC
Aug 2023	Keynote at VLDB
Aug 2023	Talk at the Simons Institute Workshop on LLMs with the recorded talk →here
Aug 2023	Talk at Duolingo
Jun 2023	Keynote at CVPR
Jun 2023	Talk at the Institute for Ethics in AI at Oxford with the recorded talk →here
May 2023	Keynote at the International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
May 2023	Talk at MPI Tubingen
May 2023	Talk at the Caltech Class on Large Language and Vision Models
May 2023	Talk at the Institute for Assured Autonomy at Johns Hopkins University
May 2023	Talk at Harvard ML Foundations Seminar Series
May 2023	SAIL Distinguished Lecture at Stanford
Apr 2023	Talk at TED with the recorded talk →here
Mar 2023	Colloquium at Berkeley
Mar 2023	Guest Lecture at the Stanford MLSys seminar co listed with Stanford CS324A: Advances in Foundation Models
Feb 2023	Guest Lecture at the Stanford CS25: Transformers United V2
Feb 2023	Talk at the Continual Causality AAAI bridge program
Jan 2023	Talk at IBM Workshop on Neuro-Symbolic AI
Jan 2023	Talk at ALPS, the NLP winter school
Dec 2022	Panel at AI Debate 3 with the recorded talk →here
Dec 2022	Talk at the NeurIPS workshop on Robustness in Sequence Modeling
Nov 2022	Talk at SoCal NLP Symposium with the recorded talk →here
Nov 2022	Talk at AI2050
Oct 2022	Colloquium at Hebrew University
Oct 2022	YanDex Distinguished Lecture at Tel-Aviv University
Oct 2022	Panel at Microsoft Research Summit on What’s next in large-scale AI
Oct 2022	Tutorial at COLING on NS4NLP: Neuro-Symbolic Modeling for NLP
Sep 2022	Keynote at Interspeech with the recorded talk →here
Sep 2022	Talk at the Simons Foundation Symposium on New Directions in Theoretical Machine Learning
Sep 2022	Talk at Penn State University
Sep 2022	Keynote at the 19th International Conference on Principles of Knowledge Representation and Reasoning (KR 2022)
July 2022	Keynote at the NAACL Workshop on Narrative Understanding
May 2022	Keynote ACL "2082: An ACL Odyssey: The Dark Matter of Language and Intelligence" and a fireside chat on "The Trajectory of ACL and the Next 60 years" with the pre-recorded talk →here
May 2022	Keynote at the ACL Workshop on Deep Learning Inside Out (DeeLIO): Knowledge Extraction and Integration for Deep Learning Architectures
Apr 2022	Panel on "Ethics, Safety, and AI: Can we have it all?" at the Center for Advancing Safety of Machine Intelligence
Mar 2022	Distinguished Lecture at Microsoft Research
Mar 2022	Keynote at the AAAI Spring Symposium on Ethical Computing: Metrics for Measuring AI’s Proficiency and Competency for Ethical Reasoning
Mar 2022	Distinguished Lecture at Max Planck Institute
Feb 2022	Keynote at WSDM
Dec 2021	Keynote at the NeurIPS workshop on Efficient Natural Language and Speech Processing (ENLSP)
Dec 2021	Keynote at the NeurIPS workshop on CtrlGen: Controllable Generative Modeling in Language and Vision
Nov 2021	Talk at the SNU AI Policy Initiative, SAPI
Nov 2021	Talk at the University of Texas Austin
Nov 2021	Keynote at the EMNLP workshop on Novel Ideas in Learning-to-Learn through Interaction (NILLI)
Oct 2021	Panel at Microsoft Research Summit on Large-scale Neural Platform Models: Opportunities, Concerns, and Directions
Oct 2021	Colloquium at UPenn
Oct 2021	Talk at Georgia Tech: "Knowledge is Power: Symbolic Knowledge Distillation, Commonsense Morality, and Multimodal Script Knowledge"
Aug 2021	Keynote at the Stanford FORUM Workshop on Universal Models: "David V.S. Goliath: the Art of Leaderboarding in the Era of Extreme-Scale Neural Models" with the recorded talk →here and the panel →here
Aug 2021	Panel at the ACL workshop on NLP for Positive Impact
Jul 2021	Distinguished Lecture at Sony
Jun 2021	Keynote at the NAACL workshop on Visually Grounded Interaction and Language (ViGIL)
Jun 2021	Keynote at the NAACL workshop on Advances in Language and Vision Research (ALVR)
May 2021	Keynote at the ICLR workshop on Workshop on Enormous Language Models (WELM): "David V.S. Goliath in the Era of Gigantic Neural Networks"
May 2021	Keynote at ICLR: "Commonsense AI: Myth and Truth"
Apr 2021	Keynote at WebConf (formerly WWW)
Mar 2021	Talk at Facebook AI
Feb 2021	Keynote at the 1st ELLIS NLP Symposium with a recorded talk →here
Feb 2021	Keynote at the AAAI Workshop on Commonsense Knowledge Graphs
Jan 2021	Teaching at the ALPS Winter School with recorded lectures →here
Jan 2021	Talk at UCL Dark Lab with a recorded talk →here
Dec 2020	Panel at AI Debate 2 at Montreal.ai with a recorded talk →here (my 3 min pitch) and →here (the whole 3.5 hr debate)
Dec 2020	Keynote at the NeurIPS Workshop on Self-Supervised Learning -- Theory and Practice
Dec 2020	Keynote at the NeurIPS Workshop on Dataset Curation and Security
Dec 2020	Talk at ByteDance
Nov 2020	Keynote at the AAAI Fall Symposium on Abstraction and Analogy
Nov 2020	Talk at the Distinguished Lecture Series at NSF CISE on "Intuitive Reasoning as (Un)supervised Neural Generation"
Nov 2020	Talk at MIT CSAIL Embodied Intelligence with a recorded talk →here
Oct 2020	Keynote at the Fall Stanford Human-Centered AI (HAI) Conference on Triangulating Intelligence: Melding Neuroscience, Psychology, and AI with a recorded talk →here (from 28:00 to 53:00)
Oct 2020	Distinguished Lecture (Department Colloquium) at Columbia University
Oct 2020	Colloquium at CMU LTI with a recorded talk →here
Oct 2020	Colloquium at University of Maryland CLIP
Oct 2020	Keynote at the Amazon ML Conference 2020
Oct 2020	Talk at Amazon AI
Sep 2020	Talk at SNU Data Science
Aug 2020	Keynote at the ECCV Workshop on Video Turing Test (VTT): Toward Human-Level Video Story Understanding with a pre-recorded talk →here
Aug 2020	Keynote at CIFAR Deep Learning Seminar Summer School with a pre-recorded talk →here
Jul 2020	ACL Tutorial on Commonsense with a recorded talk →here
Jul 2020	Keynote at the ACL Workshop on Multimodal Language
Jul 2020	Keynote at the ACL Workshop on Fact Extraction and Verification (FEVER) with a recorded talk →here
Jun 2020	CVPR tutorial on Neuro-Symbolic Visual Reasoning and Program Synthesis with a recorded talk →here
Jun 2020	Keynote at the Workshop on Symbolic-Neural Learning (SNL-2020) --- cancelled until 2021
Apr 2020	Panel at the ICLR Workshop on Bridging Cognitive Science and AI (BAICS)
Feb 2020	Keynote at the AAAI Workshop on Statistical Relational AI (StarAI)

Recent Preprints:

Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
Ali Hatamizadeh, Yejin Choi, Jan Kautz
arXiv:2605.22791

Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages
Brandon Cui, Ximing Lu, Jaehun Jung, Syeda Nahida Akter, Hyunwoo Kim, Yuxiao Qu, David Acuna, Shrimai Prabhumoye, Yejin Choi, Prithviraj Ammanabrolu
arXiv:2605.20285

ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop
Yining Hong, Jiageng Liu, Han Yin, Manling Li, Leonidas Guibas, Li Fei-Fei, Jiajun Wu, Yejin Choi
arXiv:2605.18746

NoiseRater: Meta-Learned Noise Valuation for Diffusion Model Training
Fang Wu, Haokai Zhao, Da Xing, Hanqun Cao, Tinson Xu, Yanchao Li, Xiangru Tang, Zehong Wang, Aaron Tu, Kuan Pang, Hanchen Wang, Hongbin Lin, Zeqi Zhou, Yinxi Li, Peng Xia, Li Erran Li, Molei Tao, Jure Leskovec, Aditya Joshi, Yejin Choi
arXiv:2605.08144

Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction
Zhuofeng Li, Haoxiang Zhang, Cong Wei, Pan Lu, Ping Nie, Yi Lu, Yuyang Bai, Shangbin Feng, Hangxiao Zhu, Ming Zhong, Yuyu Zhang, Jianwen Xie, Yejin Choi, James Zou, Jiawei Han, Wenhu Chen, Jimmy Lin, Dongfu Jiang, Yu Zhang
arXiv:2605.05242

How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning
Bosung Kim, Ruiyi Wang, David Acuna, Jaehun Jung, Alexander Trevithick, Brandon Cui, Yejin Choi, Prithviraj Ammanabrolu
arXiv:2605.17077

DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation
Jaehun Jung, Hyunwoo Kim, Brandon Cui, Ximing Lu, David Acuna, Prithviraj Ammanabrolu, Yejin Choi
arXiv:2605.15532

Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs
Yining Hong, Huang Huang, Manling Li, Li Fei-Fei, Leonidas Guibas, Jiajun Wu, Yejin Choi
arXiv:2602.21198

MoCo: A One-Stop Shop for Model Collaboration Research
Shangbin Feng, Yuyang Bai, Ziyuan Yang, Yike Wang, Zhaoxuan Tan, Jiajie Yan, Zhenyu Lei, Wenxuan Ding, Weijia Shi, Haojin Wang, Zhenting Qi, Yuru Jiang, Heng Wang, Chengsong Huang, Yu Fei, Jihan Yao, Yilun Du, Luke Zettlemoyer, Yejin Choi, Yulia Tsvetkov
arXiv:2601.21257

End-to-End Test-Time Training for Long Context
Arnuv Tandon, Karan Dalal, Xinhao Li, Daniel Koceja, Marcel Rød, Sam Buchanan, Xiaolong Wang, Jure Leskovec, Sanmi Koyejo, Tatsunori Hashimoto, Carlos Guestrin, Jed McCaleb, Yejin Choi, Yu Sun
arXiv:2512.23675

NVIDIA Nemotron 3: Efficient and Open Intelligence
NVIDIA: Aaron Blakeman, Aaron Grattafiori, Aarti Basant, Abhibha Gupta, Abhinav Khattar, Adi Renduchintala, Aditya Vavre, Akanksha Shukla, Akhiad Bercovich, Aleksander Ficek, Aleksandr Shaposhnikov, Alex Kondratenko, Alexander Bukharin, Alexandre Milesi, Ali Taghibakhshi, Alisa Liu, Amelia Barton, Ameya Sunil Mahabaleshwarkar, Amir Klein, Amit Zuker, Amnon Geifman, Amy Shen, Anahita Bhiwandiwalla, Andrew Tao, Anjulie Agrusa, Ankur Verma, Ann Guan, Anubhav Mandarwal, Arham Mehta, Ashwath Aithal, Ashwin Poojary, Asif Ahamed, Asit Mishra, Asma Kuriparambil Thekkumpate, Ayush Dattagupta, Banghua Zhu, Bardiya Sadeghi, Barnaby Simkin, Ben Lanir, Benedikt Schifferer, Besmira Nushi, Bilal Kartal, Bita Darvish Rouhani, Boris Ginsburg, Brandon Norick, Brandon Soubasis, Branislav Kisacanin, Brian Yu, Bryan Catanzaro, Carlo del Mundo, Chantal Hwang, Charles Wang, Cheng-Ping Hsieh, Chenghao Zhang, Chenhan Yu, Chetan Mungekar, Chintan Patel, Chris Alexiuk, Christopher Parisien, Collin Neale, Cyril Meurillon, Damon Mosk-Aoyama, Dan Su, Dane Corneil, Daniel Afrimi, Daniel Lo, Daniel Rohrer, Daniel Serebrenik, Daria Gitman, Daria Levy, Darko Stosic, David Mosallanezhad, Deepak Narayanan, Dhruv Nathawani, Dima Rekesh, Dina Yared, Divyanshu Kakwani, Dong Ahn, Duncan Riach, Dusan Stosic, Edgar Minasyan, Edward Lin, Eileen Long, Eileen Peters Long, Elad Segal, Elena Lantz, Ellie Evans, Elliott Ning, Eric Chung, Eric Harper, Eric Tramel, Erick Galinkin, Erik Pounds, Evan Briones, Evelina Bakhturina, Evgeny Tsykunov, Faisal Ladhak, Fay Wang, Fei Jia et al. (258 additional authors not shown)
arXiv:2512.20856

Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA: Aaron Blakeman, Aaron Grattafiori, Aarti Basant, Abhibha Gupta, Abhinav Khattar, Adi Renduchintala, Aditya Vavre, Akanksha Shukla, Akhiad Bercovich, Aleksander Ficek, Aleksandr Shaposhnikov, Alex Kondratenko, Alexander Bukharin, Alexandre Milesi, Ali Taghibakhshi, Alisa Liu, Amelia Barton, Ameya Sunil Mahabaleshwarkar, Amir Klein, Amit Zuker, Amnon Geifman, Amy Shen, Anahita Bhiwandiwalla, Andrew Tao, Ann Guan, Anubhav Mandarwal, Arham Mehta, Ashwath Aithal, Ashwin Poojary, Asif Ahamed, Asma Kuriparambil Thekkumpate, Ayush Dattagupta, Banghua Zhu, Bardiya Sadeghi, Barnaby Simkin, Ben Lanir, Benedikt Schifferer, Besmira Nushi, Bilal Kartal, Bita Darvish Rouhani, Boris Ginsburg, Brandon Norick, Brandon Soubasis, Branislav Kisacanin, Brian Yu, Bryan Catanzaro, Carlo del Mundo, Chantal Hwang, Charles Wang, Cheng-Ping Hsieh, Chenghao Zhang, Chenhan Yu, Chetan Mungekar, Chintan Patel, Chris Alexiuk, Christopher Parisien, Collin Neale, Damon Mosk-Aoyama, Dan Su, Dane Corneil, Daniel Afrimi, Daniel Rohrer, Daniel Serebrenik, Daria Gitman, Daria Levy, Darko Stosic, David Mosallanezhad, Deepak Narayanan, Dhruv Nathawani, Dima Rekesh, Dina Yared, Divyanshu Kakwani, Dong Ahn, Duncan Riach, Dusan Stosic, Edgar Minasyan, Edward Lin, Eileen Long, Eileen Peters Long, Elena Lantz, Ellie Evans, Elliott Ning, Eric Chung, Eric Harper, Eric Tramel, Erick Galinkin, Erik Pounds, Evan Briones, Evelina Bakhturina, Faisal Ladhak, Fay Wang, Fei Jia, Felipe Soares, Feng Chen, Ferenc Galko, Frankie Siino, Gal Hubara Agam, Ganesh Ajjanagadde, Gantavya Bhatt et al. (213 additional authors not shown)
arXiv:2512.20848

Adaptation of Agentic AI
Pengcheng Jiang, Jiacheng Lin, Zhiyi Shi, Zifeng Wang, Luxi He, Yichen Wu, Ming Zhong, Peiyang Song, Qizheng Zhang, Heng Wang, Xueqiang Xu, Hanwen Xu, Pengrui Han, Dylan Zhang, Jiashuo Sun, Chaoqi Yang, Kun Qian, Tian Wang, Changran Hu, Manling Li, Quanzheng Li, Hao Peng, Sheng Wang, Jingbo Shang, Chao Zhang, Jiaxuan You, Liyuan Liu, Pan Lu, Yu Zhang, Heng Ji, Yejin Choi, Dawn Song, Jimeng Sun, Jiawei Han
arXiv:2512.16301

RiboPO: Preference Optimization for Structure- and Stability-Aware RNA Design
Minghao Sun, Hanqun Cao, Zhou Zhang, Chen Wei, Liang Wang, Tianrui Jia, Zhiyuan Liu, Tianfan Fu, Xiangru Tang, Yejin Choi, Pheng-Ann Heng, Fang Wu, Yang Zhang
arXiv:2510.21161

L^2M^3OF: A Large Language Multimodal Model for Metal-Organic Frameworks
Jiyu Cui, Fang Wu, Haokai Zhao, Minggao Feng, Xenophon Evangelopoulos, Andrew I. Cooper, Yejin Choi
arXiv:2510.20976

DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Yejin Choi, Jan Kautz, Pavlo Molchanov
arXiv:2510.15110

From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?
Hanqun Cao, Hongrui Zhang, Junde Xu, Zhou Zhang, Lingdong Shen, Minghao Sun, Ge Liu, Jinbo Xu, Wu-Jun Li, Jinren Ni, Cesar de la Fuente-Nunez, Tianfan Fu, Yejin Choi, Pheng-Ann Heng, Fang Wu
arXiv:2510.01571

Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
Aaron Tu, Weihao Xuan, Heli Qi, Xu Huang, Qingcheng Zeng, Shayan Talaei, Yijia Xiao, Peng Xia, Xiangru Tang, Yuchen Zhuang, Bing Hu, Hanqun Cao, Wenqi Shi, Tianang Leng, Rui Yang, Yingjian Chen, Ziqi Wang, Irene Li, Nan Liu, Huaxiu Yao, Li Erran Li, Ge Liu, Amin Saberi, Naoto Yokoya, Jure Leskovec, Yejin Choi, Fang Wu6
arXiv:2509.21882

A Deep Reinforcement Learning Platform for Antibiotic Discovery
Hanqun Cao, Marcelo D. T. Torres, Jingjie Zhang, Zijun Gao, Fang Wu, Chunbin Gu, Jure Leskovec, Yejin Choi, Cesar de la Fuente-Nunez, Guangyong Chen, Pheng-Ann Heng
arXiv:2509.18153

UQ: Assessing Language Models on Unsolved Questions
Fan Nie, Ken Ziyu Liu, Zihao Wang, Rui Sun, Wei Liu, Weijia Shi, Huaxiu Yao, Linjun Zhang, Andrew Y. Ng, James Zou, Sanmi Koyejo, Yejin Choi, Percy Liang, Niklas Muennighoff
arXiv:2508.17580

Zero-shot Multimodal Document Retrieval via Cross-modal Question Generation
Yejin Choi, Jaewoo Park, Janghan Yoon, Saejin Kim, Jaehyun Jeon, Youngjae Yu
arXiv:2508.17079

NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
NVIDIA, :, Aarti Basant, Abhijit Khairnar, Abhijit Paithankar, Abhinav Khattar, Adithya Renduchintala, Aditya Malte, Akhiad Bercovich, Akshay Hazare, Alejandra Rico, Aleksander Ficek, Alex Kondratenko, Alex Shaposhnikov, Alexander Bukharin, Ali Taghibakhshi, Amelia Barton, Ameya Sunil Mahabaleshwarkar, Amy Shen, Andrew Tao, Ann Guan, Anna Shors, Anubhav Mandarwal, Arham Mehta, Arun Venkatesan , et al. (192 additional authors not shown)
arXiv:2508.14444

Advancing Science- and Evidence-based AI Policy
Rishi Bommasani, Sanjeev Arora, Jennifer Chayes, Yejin Choi, Mariano-Florentino Cuéllar, Li Fei-Fei, Daniel E. Ho, Dan Jurafsky, Sanmi Koyejo, Hima Lakkaraju, Arvind Narayanan, Alondra Nelson, Emma Pierson, Joelle Pineau, Scott Singer, Gaël Varoquaux, Suresh Venkatasubramanian, Ion Stoica, Percy Liang, Dawn Song
arXiv:2508.02748

The Invisible Leash: Why RLVR May or May Not Escape Its Origin
Fang Wu, Weihao Xuan, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, Yejin Choi
arXiv:2507.14843

Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
Mingjie Liu, Shizhe Diao, Jian Hu, Ximing Lu, Xin Dong, Hao Zhang, Alexander Bukharin, Shaokun Zhang, Jiaqi Zeng, Makesh Narsimhan Sreedhar, Gerald Shen, David Mosallanezhad, Di Zhang, Jonas Yang, June Yang, Oleksii Kuchaiev, Guilin Liu, Zhiding Yu, Pavlo Molchanov, Yejin Choi, Jan Kautz, Yi Dong
arXiv:2507.12507

The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality
Benjamin Newman, Abhilasha Ravichander, Jaehun Jung, Rui Xin, Hamish Ivison, Yegor Kuznetsov, Pang Wei Koh, Yejin Choi
arXiv:2507.08371

Resource Rational Contractualism Should Guide AI Alignment
Sydney Levine, Matija Franklin, Tan Zhi-Xuan, Secil Yanik Guyot, Lionel Wong, Daniel Kilov, Yejin Choi, Joshua B. Tenenbaum, Noah Goodman, Seth Lazar, Iason Gabriel
arXiv:2506.17434

Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases
Yubeen Bae, Minchan Kim, Jaejin Lee, Sangbum Kim, Jaehyung Kim, Yejin Choi, Niloofar Mireshghallah
arXiv:2506.17336

When to Trust Context: Self-Reflective Debates for Context Reliability
Zeqi Zhou, Fang Wu, Shayan Talaei, Haokai Zhao, Cheng Meixin, Tinson Xu, Amin Saberi, Yejin Choi
arXiv:2506.06020

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding
Jaehyun Jeon, Min Soo Kim, Jang Han Yoon, Sumin Shim, Yejin Choi, Hanbin Kim, Youngjae Yu
arXiv:2505.05026

A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage
Rui Xin, Niloofar Mireshghallah, Shuyue Stella Li, Michael Duan, Hyunwoo Kim, Yejin Choi, Yulia Tsvetkov, Sewoong Oh, Pang Wei Koh
arXiv:2504.21035

RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, Manling Li
arXiv:2504.20073

Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning
Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
arXiv:2504.13941

Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning
Ximing Lu, Seungju Han, David Acuna, Hyunwoo Kim, Jaehun Jung, Shrimai Prabhumoye, Niklas Muennighoff, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
arXiv:2504.04383

Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models
NVIDIA, :, Aaron Blakeman, Aarti Basant, Abhinav Khattar, Adithya Renduchintala, Akhiad Bercovich, Aleksander Ficek, Alexis Bjorlin, Ali Taghibakhshi, Amala Sanjay Deshmukh, Ameya Sunil Mahabaleshwarkar, Andrew Tao, Anna Shors, Ashwath Aithal, Ashwin Poojary, Ayush Dattagupta, Balaram Buddharaju, Bobby Chen, Boris Ginsburg, Boxin Wang, Brandon Norick, Brian Butterfield, Bryan Catanzaro, Carlo del Mundo , et al. (176 additional authors not shown)
arXiv:2504.03624

When One LLM Drools, Multi-LLM Collaboration Rules
Shangbin Feng, Wenxuan Ding, Alisa Liu, Zifeng Wang, Weijia Shi, Yike Wang, Zejiang Shen, Xiaochuang Han, Hunter Lang, Chen-Yu Lee, Tomas Pfister, Yejin Choi, Yulia Tsvetkov
arXiv:2502.04506

International AI Safety Report
Yoshua Bengio, Sören Mindermann, Daniel Privitera, Tamay Besiroglu, Rishi Bommasani, Stephen Casper, Yejin Choi, Philip Fox, Ben Garfinkel, Danielle Goldfarb, Hoda Heidari, Anson Ho, Sayash Kapoor, Leila Khalatbari, Shayne Longpre, Sam Manning, Vasilios Mavroudis, Mantas Mazeika, Julian Michael, Jessica Newman, Kwan Yee Ng, Chinasa T. Okolo, Deborah Raji, Girish Sastry, Elizabeth Seger , et al. (71 additional authors not shown)
arXiv:2501.17805

International Scientific Report on the Safety of Advanced AI (Interim Report)
Yoshua Bengio, Sören Mindermann, Daniel Privitera, Tamay Besiroglu, Rishi Bommasani, Stephen Casper, Yejin Choi, Danielle Goldfarb, Hoda Heidari, Leila Khalatbari, Shayne Longpre, Vasilios Mavroudis, Mantas Mazeika, Kwan Yee Ng, Chinasa T. Okolo, Deborah Raji, Theodora Skeadas, Florian Tramèr, Bayo Adekanmbi, Paul Christiano, David Dalrymple, Thomas G. Dietterich, Edward Felten, Pascale Fung, Pierre-Olivier Gourinchas , et al. (19 additional authors not shown)
arXiv:2412.05282

xGen-MM (BLIP-3): A Family of Open Large Multimodal Models
Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S Ryoo, Shrikant Kendre, Jieyu Zhang, Shaoyen Tseng, Gustavo A Lujan-Moreno, Matthew L Olson, Musashi Hinck, David Cobbley, Vasudev Lal, Can Qin, Shu Zhang, Chia-Chih Chen, Ning Yu, Juntao Tan, Tulika Manoj Awalgaonkar, Shelby Heinecke , et al. (8 additional authors not shown)
arXiv:2408.08872

Recent Publications (2016 - 2026):

HieraSuite: A Holistic Toolkit for Building Versatile System-User Instruction Hierarchy
Liwei Jiang, Erick Galinkin, Makesh Narsimhan Sreedhar, Chong Xiang, Yejin Choi, Traian Rebedea, Christopher Parisien
COLM 2026

Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs
Xingyu Fu, Siyi Liu, Yinuo Xu, Pan Lu, Guangqiuse Hu, Tianbo Yang, Taran Anantasagar, Christopher Shen, Yikai Mao, Yuanzhe Liu, Keyush Shah, Chung Un Lee, Yejin Choi, James Zou, Dan Roth, Chris Callison-Burch
COLM 2026

Data-efficient pre-training by scaling synthetic megadocs
Konwoo Kim, Suhas Kotha, Yejin Choi, Tatsunori Hashimoto, Nick Haber, Percy Liang
COLM 2026

SurfDesign: Effective Protein Design on Molecular Surfaces
Fang Wu, Shuting Jin, Xiangru Tang, Mark Gerstein, Xiangxiang Zeng, Yejin Choi, Jure Leskovec, Jinbo Xu
KDD 2026 AI4Science Track

Deciphering Fingerprints of 3D Molecular Surfaces for Accurate Epitope Prediction
Fang Wu, Weihao Xuan, Jure Leskovec, Yejin Choi, Li Erran Li
KDD 2026 AI4Science Track

Synthetic Mixed Training: Scaling Parametric Knowledge Acquisition Beyond RAG
Seungju Han, Konwoo Kim, Chanwoo Park, Benjamin Newman, Suhas Kotha, Jaehun Jung, James Zou, Yejin Choi
EMNLP 2026 Findings

FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment
Betty Xiong, Jillian Fisher, Benjamin Newman, Meng Hu, Shivangi Gupta, Yejin Choi, Lanyan Fang, Russ B Altman
EMNLP 2026

When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought
Yiyang Zhou, Haoqin Tu, Zijun Wang, Zeyu Wang, Niklas Muennighoff, Fan Nie, Yejin Choi, James Zou, Chaorui Deng, Shen Yan, Haoqi Fan, Cihang Xie, Huaxiu Yao, Qinghao Ye
CVPR 2026

VideoNet: A Large-Scale Dataset for Domain-Specific Action Recognition
Tanush Yadav, Mohammadreza Salehi, Jae Sung Park, Vivek Ramanujan, Hannaneh Hajishirzi, Yejin Choi, Ali Farhadi, Rohun Tripathi, Ranjay Krishna
CVPR 2026 -- Compute Transparency Champion

NitroGen: An Open Foundation Model for Generalist Gaming Agents
Loïc Magne, Anas Awadalla, Guanzhi Wang, Yinzhen Xu, Joshua Belofsky, Fengyuan Hu, Joohwan Kim, Ludwig Schmidt, Georgia Gkioxari, Jan Kautz, Yisong Yue, Yejin Choi, Yuke Zhu, Linxi "Jim" Fan
CVPR 2026 -- Best Paper Honorable Mention

Proteo-R1: Reasoning Foundation Models for De Novo Protein Design
Fang Wu, Weihao Xuan, Heli Qi, Hanqun Cao, Heng-Jui Chang, Zeqi Zhou, Haokai Zhao, Ma Jian, Carl Ma, Yu-Chi Cheng, Kuan Pang, Xiangru Tang, Zehong Wang, Guanlue Li, Hanchen Wang, Kejun Ying, Pan Lu, Chiho Im, Seungju Han, Peng Xia, Tinson Xu, Yinxi Li, Deyao Zhu, Pheng-Ann Heng, Naoto Yokoya, Masashi Sugiyama, Li Erran Li, Jure Leskovec, Yejin Choi
ICML 2026

RAGEN-2: Reasoning Collapse in Agentic RL
Zihan Wang, Chi Gui, Xing Jin, Qineng Wang, Licheng Liu, Kangrui Wang, Shiqi Chen, Linjie Li, Zhengyuan Yang, Pingyue Zhang, Yiping Lu, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, Manling Li
ICML 2026 Oral

Latent Collaboration in Multi-Agent Systems
Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, Ling Yang
ICML 2026 Spotlight

Learning to Discover at Test Time
Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, Jed McCaleb, Xiaolong Wang, Jan Kautz, Yejin Choi, James Zou, Carlos Guestrin, Yu Sun
ICML 2026 Spotlight

Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale
David Acuna, Chao-Han Huck Yang, Yuntian Deng, Jaehun Jung, Ximing Lu, Prithviraj Ammanabrolu, Hyunwoo Kim, Yuan-Hong Liao, Yejin Choi
ICML 2026

Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Text
Ximing Lu, David Acuna, Jaehun Jung, Jian Hu, Di Zhang, Shizhe Diao, Yunheng Zou, Shaokun Zhang, Brandon Cui, Mingjie Liu, Hyunwoo Kim, Prithviraj Ammanabrolu, Jan Kautz, Yi Dong, Yejin Choi
ICML 2026

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Peter Belcak, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Yejin Choi, Jan Kautz, Pavlo Molchanov
ICML 2026

Towards Execution-Grounded Automated AI Research
Chenglei Si, Zitong Yang, Yejin Choi, Emmanuel Candès, Diyi Yang, Tatsunori Hashimoto
ICML 2026

BroRL: Scaling Reinforcement Learning via Broadened Exploration
Jian Hu, Mingjie Liu, Ximing Lu, Fang Wu, Zaid Harchaoui, Shizhe Diao, Yejin Choi, Pavlo Molchanov, Jun Yang, Jan Kautz, Yi Dong
ICML 2026

ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
Hongjin Su, Shizhe Diao, Ximing Lu, Mingjie Liu, Jiacheng Xu, Xin Dong, Yonggan Fu, Peter Belcak, Hanrong Ye, Hongxu Yin, Yi Dong, Evelina Bakhturina, Tao Yu, Yejin Choi, Jan Kautz, Pavlo Molchanov
ICML 2026

MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
Zexue He, Yu Wang, Churan Zhi, Yuanzhe Hu, Tzu-Ping Chen, Lang Yin, Ze Chen, Tong Arthur Wu, Siru Ouyang, Zihan Wang, Jiaxin Pei, Julian McAuley, Yejin Choi, Alex Pentland
ICML 2026

Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
Mickel Liu, Liwei Jiang, Yancheng Liang, Simon Shaolei Du, Yejin Choi, Tim Althoff, Natasha Jaques
ICML 2026

Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch
Hyunwoo Kim, Niloofar Mireshghallah, Michael Duan, Rui Xin, Shuyue Stella Li, Jaehun Jung, David Acuna, Qi Pang, Hanshen Xiao, G. Edward Suh, Sewoong Oh, Yulia Tsvetkov, Pang Wei Koh, Yejin Choi
ICML 2026

Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations
Linjie Li, Mahtab Bigverdi, Jiawei Gu, Zixian Ma, Yinuo Yang, Ziang Li, Yejin Choi, Ranjay Krishna
ICLR 2026

ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
Jiawei Gu, Yunzhuo Hao, Huichen Will Wang, Linjie Li, Michael Qizhe Shieh, Yejin Choi, Ranjay Krishna, Yu Cheng
ICLR 2026

MoReBench: Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes
Yu Ying Chiu, Michael S. Lee, Rachel Calcott, Brandon Handoko, Paul de Font-Reaulx, Paula Rodriguez, Chen Bo Calvin Zhang, Ziwen Han, Udari Madhushani Sehwag, Yash Maurya, Christina Q Knight, Harry R. Lloyd, Florence Bacus, Mantas Mazeika, Bing Liu, Yejin Choi, Mitchell L Gordon, Sydney Levine
ICLR 2026

Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas
Yu Ying Chiu, Zhilin Wang, Sharan Maiya, Yejin Choi, Kyle Fish, Sydney Levine, Evan Hubinger
ICLR 2026

ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge
Zhilin Wang, Jaehun Jung, Ximing Lu, Shizhe Diao, Ellie Evans, Jiaqi Zeng, Pavlo Molchanov, Yejin Choi, Jan Kautz, Yi Dong
ICLR 2026

Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability
Taylor Sorensen, Benjamin Newman, Jared Moore, Chan Park, Jillian Fisher, Niloofar Mireshghallah, Liwei Jiang, Yejin Choi
ICLR 2026

In-the-Flow Agentic System Optimization for Effective Planning and Tool Use
Zhuofeng Li, Haoxiang Zhang, Seungju Han, Sheng Liu, Jianwen Xie, Yu Zhang, Yejin Choi, James Zou, Pan Lu
ICLR 2026

Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data
Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, Bryan Catanzaro
ICLR 2026

Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction
Yong Lin, Shange Tang, Bohan Lyu, Ziran Yang, Jui-Hui Chung, Haoyu Zhao, Lai Jiang, Yihan Geng, Jiawei Ge, Jingruo Sun, Jiayun Wu, Jiri Gesi, Ximing Lu, David Acuna, Kaiyu Yang, Hongzhou Lin, Yejin Choi, Danqi Chen, Sanjeev Arora, Chi Jin
ICLR 2026

DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
Fang Wu, Weihao Xuan, Heli Qi, Ximing Lu, Aaron Tu, Li Erran Li, Yejin Choi
ICLR 2026

Multiplayer Nash Preference Optimization
Fang Wu, Xu Huang, Weihao Xuan, Zhiwei Zhang, Yijia Xiao, Guancheng Wan, Xiaomin Li, Bing Hu, Peng Xia, Jure Leskovec, Yejin Choi
ICLR 2026

RLP: Reinforcement as a Pretraining Objective
Ali Hatamizadeh, Syeda Nahida Akter, Shrimai Prabhumoye, Jan Kautz, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
ICLR 2026

PoseX: AI Defeats Physics-based Methods on Protein Ligand Cross-Docking
Yize Jiang, Xinze Li, Yuanyuan Zhang, Jin Han, Youjun Xu, Ayush Pandit, ZAIXI ZHANG, Mengdi Wang, Mengyang Wang, Chong Liu, Guang Yang, Yejin Choi, Yingzhou Lu, Wu-Jun Li, Tianfan Fu, Fang Wu, Junhong Liu
ICLR 2026

Activation Steering for LLM Alignment via a Unified ODE-Based Framework
Hongjue Zhao, Haosen Sun, Jiangtao Kong, Xiaochang Li, Qineng Wang, Liwei Jiang, Qi Zhu, Tarek F. Abdelzaher, Yejin Choi, Manling Li, Huajie Shao
ICLR 2026

SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs
Yuling Gu, Oyvind Tafjord, Hyunwoo Kim, Jared Moore, Ronan Le Bras, Peter Clark, Yejin Choi
ICLR 2026

What Lies Beyond the View? Actively Constructing Spatial Beliefs in Foundation Models
Pingyue Zhang, Zihan Huang, Yue Wang, Jieyu Zhang, Letian Xue, Zihan Wang, Qineng Wang, Keshigeyan Chandrasegaran, Ruohan Zhang, Yejin Choi, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Manling Li
ICLR 2026

OpenThoughts: Data Recipes for Reasoning Models
Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng , et al. (25 additional authors not shown)
ICLR 2026

Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy, Research, and Practice
A. Feder Cooper, Christopher A. Choquette-Choo, Miranda Bogen, Matthew Jagielski, Katja Filippova, Ken Ziyu Liu, Alexandra Chouldechova, Jamie Hayes, Yangsibo Huang, Niloofar Mireshghallah, Ilia Shumailov, Eleni Triantafillou, Peter Kairouz, Nicole Mitchell, Percy Liang, Daniel E. Ho, Yejin Choi, Sanmi Koyejo, Fernando Delgado, James Grimmelmann, Vitaly Shmatikov, Christopher De Sa, Solon Barocas, Amy Cyphert, Mark Lemley , et al. (10 additional authors not shown)
NeurIPS 2025 Position Paper Track Oral

Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
Yiyou Sun, Yu Gai, Lijie Chen, Abhilasha Ravichander, Yejin Choi, Dawn Song
NeurIPS 2025

ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, Yi Dong
NeurIPS 2025

VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
Kangrui Wang, Pingyue Zhang, Zihan Wang, Yaning Gao, Linjie Li, Qineng Wang, Hanyang Chen, Yiping Lu, Zhengyuan Yang, Lijuan Wang, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Yejin Choi, Manling Li
NeurIPS 2025

AI Debate Aids Assessment of Controversial Claims
Salman Rahman, Sheriff Issaka, Ashima Suvarna, Genglin Liu, James Shiffer, Jaeyoung Lee, Md Rizwan Parvez, Hamid Palangi, Shi Feng, Nanyun Peng, Yejin Choi, Julian Michael, Liwei Jiang, Saadia Gabriel
NeurIPS 2025

Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostafa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
NeurIPS 2025 Spotlight

Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
Brian Siyuan Zheng, Alisa Liu, Orevaoghene Ahia, Jonathan Hayase, Yejin Choi, Noah A. Smith
NeurIPS 2025 Spotlight

Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)
Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Maarten Sap, Yulia Tsvetkov, Nouha Dziri, Yejin Choi
NeurIPS 2025 Datasets and Benchmarks -- Best Paper Award

Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions
David Acuna, Ximing Lu, Jaehun Jung, Hyunwoo Kim, Amlan Kar, Sanja Fidler, Yejin Choi
EMNLP 2025

Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index
Hao Xu, Jiacheng Liu, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi
EMNLP 2025 -- Best Paper Award

Making VLMs More Robot-Friendly: Self-Critical Distillation of Low-Level Procedural Reasoning
Chan Young Park, Jillian Fisher, Marius Memmel, Dipika Khullar, Seoho Yun, Abhishek Gupta, Yejin Choi
EMNLP 2025

Challenge on LLMs for Embodied Agents
Manling Li, Tianwei Bao, Qineng Wang, Yu Zhou, Shiyu Zhao, Kangrui Wang, Jiayuan Mao, Ruohan Zhang, Weiyu Liu, Tony Lee, Li Erran Li, Yejin Choi, Percy Liang, Li Fei-Fei, Jiajun Wu
NeurIPS 2025 Competition Track

Multi-Attribute Constraint Satisfaction via Language Model Rewriting
Ashutosh Baheti, Debanjana Chakraborty, Faeze Brahman, Ronan Le Bras, Ximing Lu, Nouha Dziri, Yejin Choi, Mark Riedl, Maarten Sap
TMLR 2025

HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
Abhilasha Ravichander, Shrusti Ghela, David Wadden, Yejin Choi
ACL 2025 -- Outstanding Paper Award

Biased LLMs can Influence Political Decision-Making
Jillian Fisher, Shangbin Feng, Robert Aron, Thomas Richardson, Yejin Choi, Daniel W Fisher, Jennifer Pan, Yulia Tsvetkov, Katharina Reinecke
ACL 2025

Can Language Models Reason about Individualistic Human Values and Preferences?
Liwei Jiang, Taylor Sorensen, Sydney Levine, Yejin Choi
ACL 2025

CulturalBench: A Robust, Diverse and Challenging Benchmark for Measuring LMs' Cultural Knowledge Through Human-AI Red-Teaming
Yu Ying Chiu, Liwei Jiang, Bill Yuchen Lin, Chan Young Park, Shuyue Stella Li, Sahithya Ravi, Mehar Bhatia, Maria Antoniak, Yulia Tsvetkov, Vered Shwartz, Yejin Choi
ACL 2025

OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens
Jiacheng Liu, Taylor Blanton, Yanai Elazar, Sewon Min, YenSung Chen, Arnavi Chheda-Kothary, Huy Tran, Byron Bischoff, Eric Marsh, Michael Schmitz, Cassidy Trier, Aaron Sarnat, Jenna James, Jon Borchardt, Bailey Kuehl, Evie Cheng, Karen Farley, Sruthi Sreeram, Taira Anderson, David Albright, Carissa Schoenick, Luca Soldaini, Dirk Groeneveld, Rock Yuren Pang, Pang Wei Koh, Noah A. Smith, Sophie Lebrecht, Yejin Choi, Hannaneh Hajishirzi, Ali Farhadi, Jesse Dodge
ACL 2025 Demonstration Track -- Best Demo Paper Award

LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferences
Yusuke Hirota, Boyi Li, Ryo Hachiuma, Yueh-Hua Wu, Boris Ivanovic, Yuta Nakashima, Marco Pavone, Yejin Choi, Yu-Chiang Frank Wang, Chao-Han Huck Yang
ACL 2025 Industry Track

Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models
Hyunwoo Kim, Melanie Sclar, Tan Zhi-Xuan, Lance Ying, Sydney Levine, Yang Liu, Joshua B. Tenenbaum, Yejin Choi
COLM 2025

SuperBPE: Space Travel for Language Models
Alisa Liu, Jonathan Hayase, Valentin Hofmann, Sewoong Oh, Noah A. Smith, Yejin Choi
COLM 2025

Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers
Wooseok Seo, Seungju Han, Jaehun Jung, Benjamin Newman, Seungwon Lim, Seungbeen Lee, Ximing Lu, Yejin Choi, Youngjae Yu
COLM 2025

LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception
Yuan-Hong Liao, Sven Elflein, Liu He, Laura Leal-Taixé, Yejin Choi, Sanja Fidler, David Acuna
COLM 2025

HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Interactive AI Agents
Xuhui Zhou, Hyunwoo Kim, Faeze Brahman, Liwei Jiang, Hao Zhu, Ximing Lu, Frank Xu, Bill Yuchen Lin, Yejin Choi, Niloofar Mireshghallah, Ronan Le Bras, Maarten Sap
COLM 2025

X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
Salman Rahman, Liwei Jiang, James Shiffer, Genglin Liu, Sheriff Issaka, Md Rizwan Parvez, Hamid Palangi, Kai-Wei Chang, Yejin Choi, Saadia Gabriel
COLM 2025

The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage
Skyler Hallinan, Jaehun Jung, Melanie Sclar, Ximing Lu, Abhilasha Ravichander, Sahana Ramnath, Yejin Choi, Sai Praneeth Karimireddy, Niloofar Mireshghallah, Xiang Ren
COLM 2025

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang et al.
TMLR 2025

RESTOR: Knowledge Recovery in Machine Unlearning
Keivan Rezaei, Khyathi Chandu, Soheil Feizi, Yejin Choi, Faeze Brahman, Abhilasha Ravichander
TMLR 2025

Bias in Gender Bias Benchmarks: How Confounding Features Distort Evaluation
Yusuke Hirota, Ryo Hachiuma, Boyi Li, Ximing Lu, Michael Ross Boone, Boris Ivanovic, Yejin Choi, Marco Pavone, Yu-Chiang Frank Wang, Noa Garcia, Yuta Nakashima, Chao-Han Huck Yang
ICCV 2025

Position: Political Neutrality in AI is Impossible— But Here’s How to Approximate It
Jillian Fisher, Ruth Elisabeth Appel, Chan Young Park, Yujin Potter, Liwei Jiang, Taylor Sorensen, Shangbin Feng, Yulia Tsvetkov, Margaret Roberts, Jennifer Pan, Dawn Song, Yejin Choi
ICML 2025 Position Track

SafetyAnalyst: Interpretable, transparent, and steerable safety moderation for AI behavior
Jing-Jing Li, Valentina Pyatkin, Max Kleiman-Weiner, Liwei Jiang, Nouha Dziri, Anne Collins, Jana Schaich Borg, Maarten Sap, Yejin Choi, Sydney Levine
ICML 2025

Diverging Preferences: When do Annotators Disagree and do Models Know?
Michael JQ Zhang, Zhilin Wang, Jena D. Hwang, Yi Dong, Olivier Delalleau, Yejin Choi, Eunsol Choi, Xiang Ren, Valentina Pyatkin
ICML 2025

ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, Yejin Choi
ICML 2025

Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence
Shangbin Feng, Zifeng Wang, Yike Wang, Sayna Ebrahimi, Hamid Palangi, Lesly Miculicich, Achin Kulshrestha, Nathalie Rauschmayr, Yejin Choi, Yulia Tsvetkov, Chen-Yu Lee, Tomas Pfister
ICML 2025

Investigating machine moral judgement through the Delphi experiment
Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny T. Liang, Sydney Levine, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jack Hessel, Jon Borchardt, Taylor Sorensen, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini and Yejin Choi
Nature Machine Intelligence 2025

LLMs' Potential Influences on Our Democracy: Challenges and Opportunities
Yujin Potter, David Rand, Yejin Choi, Dawn Song
ICLR 2025 Blogpost track

Synthetic Visual Genome
Jae Sung Park, Zixian Ma, Linjie Li, Chenhao Zheng, Cheng-Yu Hsieh, Ximing Lu, Khyathi Chandu, Quan Kong, Norimasa Kobori, Ali Farhadi, Yejin Choi, Ranjay Krishna
CVPR 2025

One-Minute Video Generation with Test-Time Training
Jiarui Xu, Shihao Han, Karan Dalal, Daniel Koceja, Yue Zhao, Ka Chun Cheung, Yejin Choi, Jan Kautz, Yu Sun, Xiaolong Wang
CVPR 2025

Language Model Alignment in Multilingual Trolley Problems
Zhijing Jin, Max Kleiman-Weiner, Giorgio Piatti, Sydney Levine, Jiarui Liu, Fernando Gonzalez Adauto, Francesco Ortu, András Strausz, Mrinmaya Sachan, Rada Mihalcea, Yejin Choi, Bernhard Schölkopf
ICLR 2025 Spotlight

Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement
Jaehun Jung, Faeze Brahman, Yejin Choi
ICLR 2025 Oral

AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text
Ximing Lu, Melanie Sclar, Skyler Hallinan, Niloofar Mireshghallah, Jiacheng Liu, Seungju Han, Allyson Ettinger, Liwei Jiang, Khyathi Chandu, Nouha Dziri, Yejin Choi
ICLR 2025 Oral

CertainlyUncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
Khyathi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, Jae Sung Park, Jack Hessel, Lijuan Wang, Yejin Choi
ICLR 2025

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, Bill Yuchen Lin
ICLR 2025

WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, Yejin Choi
ICLR 2025 Spotlight

Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset
Yingzi Ma, Jiongxiao Wang, Fei Wang, Siyuan Ma, Jiazhao Li, Jinsheng Pan, Xiujun Li, Furong Huang, Lichao Sun, Bo Li, Yejin Choi, Muhao Chen, Chaowei Xiao
ICLR 2025

Explore Theory of Mind: program-guided adversarial data generation for theory of mind reasoning
Melanie Sclar, Jane Dwivedi-Yu, Maryam Fazel-Zarandi, Yulia Tsvetkov, Yonatan Bisk, Yejin Choi, Asli Celikyilmaz
ICLR 2025

DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily Life
Yu Ying Chiu, Liwei Jiang, Yejin Choi
ICLR 2025 Spotlight

Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models
Abhilasha Ravichander, Jillian Fisher, Taylor Sorensen, Ximing Lu, Maria Antoniak, Bill Yuchen Lin, Niloofar Mireshghallah, Chandra Bhagavatula, Yejin Choi
NAACL 2025

Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs
Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, Santu Rana
NAACL 2025

RewardBench: Evaluating Reward Models for Language Modeling
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, Lester James Validad Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi
NAACL 2025 Findings

Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Korinek, Jose Hernandez-Orallo, Lewis Hammond, Eric Bigelow, Alexander Pan, Lauro Langosco, Tomasz Korbak, Heidi Zhang, Ruiqi Zhong, Seán Ó hÉigeartaigh, Gabriel Recchia, Giulio Corsi, Alan Chan, Markus Anderljung, Lilian Edwards, Aleksandar Petrov, Christian Schroeder de Witt, Sumeet Ramesh Motwan, Yoshua Bengio, Danqi Chen, Philip H.S. Torr, Samuel Albanie, Tegan Maharaj, Jakob Foerster, Florian Tramer, He He, Atoosa Kasirzadeh, Yejin Choi, David Krueger
TMLR 2024

WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models
Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Nouha Dziri, Yejin Choi
NeurIPS 2024

Data Mixture Inference Attack: BPE Tokenizers Reveal Training Data Compositions
Jonathan Hayase, Alisa Liu, Yejin Choi, Sewoong Oh, Noah A. Smith
NeurIPS 2024

Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback
Hamish Ivison, Yizhong Wang, Jiacheng Liu, Zeqiu Wu, Valentina Pyatkin, Nathan Lambert, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi
NeurIPS 2024

WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences
Yujie Lu, Dongfu Jiang, Wenhu Chen, William Yang Wang, Yejin Choi, Bill Yuchen Lin
NeurIPS 2024 Datasets and Benchmarks Track

ActionAtlas: A VideoQA Benchmark for Fine-grained Action Recognition
Mohammadreza Salehi, Jae Sung Park, Aditya Kusupati, Ranjay Krishna, Yejin Choi, Hannaneh Hajishirzi, Ali Farhadi
NeurIPS 2024 Datasets and Benchmarks Track

WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Nouha Dziri, Yejin Choi
NeurIPS 2024 Datasets and Benchmarks Track

The Art of Saying No: Contextual Noncompliance in Language Models
Faeze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, Yulia Tsvetkov, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi
NeurIPS 2024 Datasets and Benchmarks Track

MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens
Anas Awadalla, Le Xue, Oscar Lo, Manli Shu, Hannah Lee, Etash Kumar Guha, Sheng Shen, Mohamed Awadalla, Silvio Savarese, Caiming Xiong, Ran Xu, Yejin Choi, Ludwig Schmidt
NeurIPS 2024 Datasets and Benchmarks Track

Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration
Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, Yulia Tsvetkov
EMNLP 2024

CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation
Tong Chen, Akari Asai, Niloofar Mireshghallah, Sewon Min, James Grimmelmann, Yejin Choi, Hannaneh Hajishirzi, Luke Zettlemoyer, Pang Wei Koh
EMNLP 2024

Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models
Chani Jung, Dongkwan Kim, Jiho Jin, Jiseon Kim, Yeon Seonwoo, Yejin Choi, Alice Oh, Hyunwoo Kim
EMNLP 2024

StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements
Jillian Fisher, Skyler Hallinan, Ximing Lu, Mitchell L Gordon, Zaid Harchaoui, Yejin Choi
EMNLP 2024

How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models
Jaeyoung Lee, Ximing Lu, Jack Hessel, Faeze Brahman, Youngjae Yu, Yonatan Bisk, Yejin Choi, Saadia Gabriel
EMNLP 2024

In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search
Huihan Li, Yuting Ning, Zeyi Liao, Siyuan Wang, Xiang Lorraine Li, Ximing Lu, Wenting Zhao, Faeze Brahman, Yejin Choi, Xiang Ren
EMNLP 2024

Symbolic Working Memory Enhances Language Models for Complex Rule Application
Siyuan Wang, zhongyu wei, Yejin Choi, Xiang Ren
EMNLP 2024

Information-Theoretic Distillation for Reference-less Summarization
Jaehun Jung, Ximing Lu, Liwei Jiang, Faeze Brahman, Peter West, Pang Wei Koh, Yejin Choi
COLM 2024

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens
Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, Hannaneh Hajishirzi
COLM 2024

Tuning Language Models by Proxy
Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, Noah A. Smith
COLM 2024

Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild
Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, Golnoosh Farnadi
COLM 2024

Do Membership Inference Attacks Work on Large Language Models?
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, Hannaneh Hajishirzi
COLM 2024

CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting
Huihan Li, Liwei Jiang, Nouha Dziri, Xiang Ren, Yejin Choi
COLM 2024

Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
Jiacheng Liu, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi, Asli Celikyilmaz
COLM 2024

Position Paper: A Roadmap to Pluralistic Alignment
Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, Yejin Choi
ICML 2024

Structured Chemistry Reasoning with Large Language Models
Siru Ouyang, Zhuosheng Zhang, Bing Yan, Xuan Liu, Yejin Choi, Jiawei Han, Lianhui Qin
ICML 2024

Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin
ACL 2024

Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs
Siyuan Wang, zhongyu wei, Yejin Choi, Xiang Ren
ACL 2024

Impossible Distillation: for Paraphrasing and Summarization: How to Make High-quality Lemonade out of Small, Low-quality Model
Jaehun Jung, Peter West, Liwei Jiang, Faeze Brahman, Ximing Lu, Jillian Fisher, Taylor Sorensen, Yejin Choi
NAACL 2024

MacGyver: Are Large Language Models Creative Problem Solvers?
Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas L. Griffiths, Faeze Brahman
NAACL 2024

JAMDEC: Unsupervised Authorship Obfuscation using Constrained Decoding over Small Language Models
Jillian Fisher, Ximing Lu, Jaehun Jung, Liwei Jiang, Zaid Harchaoui, Yejin Choi
NAACL 2024

UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations
Wenting Zhao, Justin T Chiu, Jena D. Hwang, Faeze Brahman, Jack Hessel, Sanjiban Choudhury, Yejin Choi, Xiang Lorraine Li, Alane Suhr
NAACL 2024

NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge
Phillip Howard, Junlin Wang, Vasudev Lal, Gadi Singer, Yejin Choi, Swabha Swayamdipta
NAACL 2024 Findings

Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, Vered Shwartz
EACL 2024

Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Taylor Sorensen, Liwei Jiang, Jena Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas, Yejin Choi
AAAI 2024

The Generative AI Paradox: “What It Can Create, It May Not Understand”
*Peter West, *Ximing Lu, *Nouha Dziri, *Faeze Brahman, *Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
ICLR 2024

The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, Yejin Choi
ICLR 2024

WildChat: 1M ChatGPT Interaction Logs in the Wild
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, Yuntian Deng
ICLR 2024 Spotlight

Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement
Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, Xiang Ren
ICLR 2024 Oral

Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, Yejin Choi
ICLR 2024 Spotlight

Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, Alane Suhr
ICLR 2024

PlaSma: Procedural Knowledge Models for Language-based Planning and Re-Planning
Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, Jena D. Hwang, Xiang Lorraine Li, Hirona J. Arai, Soumya Sanyal, Keisuke Sakaguchi, Xiang Ren, Yejin Choi
ICLR 2024

Tailoring Self-Rationalizers with Multi-Reward Distillation
Sahana Ramnath, Brihi Joshi, Skyler Hallinan, Ximing Lu, Liunian Harold Li, Aaron Chan, Jack Hessel, Yejin Choi, Xiang Ren
ICLR 2024

Faith and Fate: Limits of Transformers on Compositionality
*Nouha Dziri, *Ximing Lu, *Melanie Sclar, +Xiang Lorraine Li, +Liwei Jiang, +Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Sean Welleck, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, Yejin Choi
NeurIPS 2023 Spotlight

SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, Xiang Ren
NeurIPS 2023 Spotlight

Localized Symbolic Knowledge Distillation for Visual Commonsense Models
Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Qiuyuan Huang, Peter West, Jianfeng Gao, Ali Farhadi, Yejin Choi
NeurIPS 2023

Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text
Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, Yejin Choi
NeurIPS 2023 Datasets and Benchmarks Track

RealTime QA: What's the Answer Right Now?
Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, Kentaro Inui
NeurIPS 2023 Datasets and Benchmarks Track
*Project

🍺 Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning
Ximing Lu, Faeze Brahman, Peter West, Jaehun Jang, Khyathi Chandu, Abhilasha Ravichander, Lianhui Qin, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Yuchen Lin, Skyler Hallinan, Xiang Ren, Sean Welleck, Yejin Choi
EMNLP 2023

Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statements
Jiacheng Liu, Wenya Wang, Dianzhuo Wang, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi
EMNLP 2023

Crystal: Introspective Reasoners Reinforced with Self-Feedback/a>
Jiacheng Liu, Ramakanth Pasunuru, Hannaneh Hajishirzi, Yejin Choi, Asli Celikyilmaz
EMNLP 2023

We're Afraid Language Models Aren't Modeling Ambiguity
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, Yejin Choi
EMNLP 2023

FANToM: A Benchmark for Stress-Testing Theory of Mind in Interactions
Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, Maarten Sap
EMNLP 2023

SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi
EMNLP 2023 -- Outstanding Paper Award

Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms
Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang, Jiwan Chung, Yejin Son, Yejin Choi, Youngjae Yu
EMNLP 2023

NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation
Peter West, Ronan Le Bras, Taylor Sorensen, Bill Yuchen Lin, Liwei Jiang, Ximing Lu, Khyathi Chandu, Jack Hessel, Ashutosh Baheti, Chandra Bhagavatula, Yejin Choi
EMNLP 2023 Findings

What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations
Kavel Rao, Liwei Jiang, Valentina Pyatkin, Yuling Gu, Niket Tandon, Nouha Dziri, Faeze Brahman, Yejin Choi
EMNLP 2023 Findings

"You Are An Expert Linguistic Annotator": Limits of LLMs as Analyzers of Abstract Meaning Representation
Allyson Ettinger, Jena D. Hwang, Valentina Pyatkin, Chandra Bhagavatula, Yejin Choi
EMNLP 2023 Findings

STEER: Unified Style Transfer with Expert Reinforcement
Skyler Hallinan, Faeze Brahman, Ximing Lu, Jaehun Jung, Sean Welleck, Yejin Choi
EMNLP 2023 Findings

BotPercent: Estimating Bot Populations in Twitter Communities
Zhaoxuan Tan, Shangbin Feng, Melanie Sclar, Herun Wan, Minnan Luo, Yejin Choi, Yulia Tsvetkov
EMNLP 2023 Findings

CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videos
Seungju Han, Jack Hessel, Nouha Dziri, Yejin Choi, Youngjae Yu
ICCV 2023

Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling
Kolby Nottingham, Prithviraj Ammanabrolu, Alane Suhr, Yejin Choi, Hannaneh Hajishirzi, Sameer Singh, Roy Fox
ICML 2023

Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker
Melanie Sclar, Sachin Kumar, Peter West, Alane Suhr, Yejin Choi and Yulia Tsvetkov
ACL 2023 -- Outstanding Paper Award

Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step
Liunian Harold Li, Jack Hessel, Youngjae Yu, Xiang Ren, Kai-Wei Chang and Yejin Choi
ACL 2023

Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest
Jack Hessel, Ana Marasović, Jena D. Hwang, Lillian Lee, Jeff Da, Rowan Zellers, Robert Mankoff, Yejin Choi
ACL 2023 -- Best Paper Award

ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations
Valentina Pyatkin, Jena D. Hwang, Vivek Srikumar, Ximing Lu, Liwei Jiang, Yejin Choi, Chandra Bhagavatula
ACL 2023

I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation
Chandra Bhagavatula, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, Yejin Choi
ACL 2023

SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human-Machine Collaboration
Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Meeyoung Cha, Yejin Choi, BYOUNGPIL KIM, Gunhee Kim, Eun-Ju Lee, Yong Lim, Alice Oh, Sangchul Park and Jung-Woo Ha
ACL 2023

An AI Dungeon Master's Guide: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
Pei Zhou, Andrew Zhu, Jennifer Hu, Jay Pujara, Xiang Ren, Chris Callison-Burch, Yejin Choi and Prithviraj Ammanabrolu
ACL 2023

Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts
Skyler Hallinan, Alisa Liu, Yejin Choi and Maarten Sap
ACL 2023

Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-text Rationales
Brihi Joshi, Ziyi Liu, Sahana Ramnath, Aaron Chan, Zhewei Tong, Shaoliang Nie, Qifan Wang, Yejin Choi and Xiang Ren
ACL 2023

From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models
Julia Mendelsohn, Ronan Le Bras, Yejin Choi and Maarten Sap
ACL 2023

Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation
Kung-Hsiang Huang, Kathleen McKeown, Preslav Nakov, Yejin Choi and Heng Ji
ACL 2023

REV: Information-Theoretic Evaluation of Free-Text Rationales
Hanjie Chen, Faeze Brahman, Xiang Ren, Yangfeng Ji, Yejin Choi and Swabha Swayamdipta
ACL 2023

Commonsense Knowledge Transfer for Pre-trained Language Models
Wangchunshu Zhou, Ronan Le Bras and Yejin Choi
ACL 2023 Findings

Modularized Encoder-Decoder Models for Flexible Sequence-to-Sequence Compression
Wangchunshu Zhou, Ronan Le Bras and Yejin Choi
ACL 2023 Findings

Multimodal Knowledge Alignment with Reinforcement Learning
Youngjae Yu, Jiwan Chung, Heeseung Yun, Jack Hessel, Jae Sung Park, Ximing Lu, Rowan Zellers, Prithviraj Ammanabrolu, Ronan Le Bras, Gunhee Kim, Yejin Choi
CVPR 2023

Influence Diagnostics under Self-concordance
Jillian Fisher, Lang Liu, Krishna Pillutla, Yejin Choi, and Zaid Harchaoui
AISTAT 2023

Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptions
Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen McKeown, Doug Downey, Yejin Choi
EACL 2023

Generating Sequences by Learning to [Self-]Correct
Sean Welleck*, Ximing Lu*, Peter West+, Faeze Brahman+, Tianxiao Shen, Daniel Khashabi, Yejin Choi
ICLR 2023

Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kianté Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi, Yejin Choi
ICLR 2023
*Project

🍏 The Curious Case of Commonsense Intelligence
Yejin Choi
Daedalus's special issue on AI & Society, 2022

Quark: Controllable Text Generation with Reinforced Unlearning
Ximing Lu, Sean Welleck, Liwei Jiang, Jack Hessel, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, Yejin Choi
NeurIPS 2022

COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics
Lianhui Qin, Sean Welleck, Daniel Khashabi, Yejin Choi
NeurIPS 2022

NaturalProver: Grounded Mathematical Proof Generation with Language Models
Sean Welleck, Jiacheng Liu, Ximing Lu, Hannaneh Hajishirzi, Yejin Choi
NeurIPS 2022

Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation
Melanie Sclar, Peter West, Sachin Kumar, Yulia Tsvetkov and Yejin Choi
EMNLP 2022

Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras and Yejin Choi
EMNLP 2022

Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering
Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, Yejin Choi
EMNLP 2022

Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs
Maarten Sap, Ronan Le Bras, Daniel Fried and Yejin Choi
EMNLP 2022

ProsocialDialog: A Prosocial Backbone for Conversational Agents
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi and Maarten Sap
EMNLP 2022
*Data

Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ Tasks
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddhartha Mishra, Sujan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi and Daniel Khashabi
EMNLP 2022
*Project

Twist Decoding: Diverse Generators Guide Each Other
Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, Noah A. Smith
EMNLP 2022

Toward Reproducible and Standardized Human Evaluation for Text Generation
Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith and Daniel Weld
EMNLP 2022
*Project

WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
Alisa Liu, Swabha Swayamdipta, Noah A. Smith, Yejin Choi
EMNLP 2022 Findings

NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation
Phillip Howard, Gadi Singer, Vasudev Lal, Yejin Choi and Swabha Swayamdipta
EMNLP 2022 Findings

NaturalAdversaries: Can Natural Adversaries Be As Effective As Artificial Adversaries?
Saadia Gabriel, Hamid Palangi and Yejin Choi
EMNLP 2022 Findings

The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning
Jack Hessel, Jena D. Hwang, Jae Sung Park, Rowan Zellers, Chandra Bhagavatula, Anna Rohrbach, Kate Saenko, Yejin Choi
ECCV 2022

Information-Theoretic Measures of Dataset Difficulty
Kawin Ethayarajh, Yejin Choi, Swabha Swayamdipta
ICML 2022 -- Outstanding Paper Award

MERLOT Reserve: Multimodal Neural Script Knowledge through Vision and Language and Sound
Rowan Zellers, Jiasen Lu, Ximing Lu, Youngjae Yu, Yanpeng Zhao, Mohammadreza Salehi, Aditya Kusupati, Jack Hessel, Ali Farhadi, Yejin Choi
CVPR 2022
*Project

Quantifying the Narrative Flow of Imagined versus Autobiographical Stories
Maarten Sap, Anna Jafarpour, Yejin Choi, Noah A. Smith, James W. Pennebaker, Eric Horvitz
PNAS 2022

It’s not Rocket Science: Interpreting Figurative Language in Narratives
Tuhin Chakrabarty , Yejin Choi and Vered Shwartz
TACL 2022

Aligning to Social Norms and Values in Interactive Narratives
Prithviraj Ammanabrolu, Liwei Jiang, Maarten Sap, Hanna Hajishirzi, Yejin Choi
NAACL 2022

Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
Peter West, Chandra Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, Yejin Choi
NAACL 2022

NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics
Ximing Lu, *Sean Welleck, *Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah Smith, Yejin Choi
NAACL 2022 -- Best Paper Award

Promp Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts
Daniel Khashabi, Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sameer Singh, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal and Yejin Choi
NAACL 2022

Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Lavinia Dunagan, Jacob Morrison, Alexander R. Fabbri, Yejin Choi, Noah A. Smith
NAACL 2022

Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection
Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, Noah A. Smith
NAACL 2022

Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer
Yanpeng Zhao, Jack Hessel, Youngjae Yu, Ximing Lu, Rowan Zellers, Yejin Choi
NAACL 2022

Transparent Human Evaluation for Image Captioning
Jungo Kasai, Keisuke Sakaguchi, Lavinia Dunagan, Jacob Morrison, Ronan Le Bras, Yejin Choi, Noah A. Smith
NAACL 2022

Reframing Human-AI Collaboration for Generating Free-Text Explanations
Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, Yejin Choi
NAACL 2022

Exposing the Limits of Video-Text Models through Contrast Sets
Jae Sung Park, Sheng Shen, Ali Farhadi, Trevor Darrell, Yejin Choi, Anna Rohrbach
NAACL 2022

🎃 Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text
Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah A. Smith, Yejin Choi
ACL 2022

Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlines
Saadia Gabriel, Skyler Hallinan, Maarten Sap, Pemi Nguyen, Franziska Roesner, Eunsol Choi, Yejin Choi
ACL 2022

Generated Knowledge Prompting for Commonsense Reasoning
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi
ACL 2022

Reframing Instructional Prompts to GPTk's Language
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, Hannaneh Hajishirzi
ACL 2022 Findings

Probing Factually Grounded Content Transfer with Factual Ablation
Peter West, Chris Quirk, Michel Galley, Yejin Choi
ACL 2022 Findings

Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics
Sean Welleck, Peter West, Jize Cao, Yejin Choi
AAAI 2022

🍷MERLOT: Multimodal Neural Script Knowledge Models
Rowan Zellers*, Ximing Lu*, Jack Hessel*, Youngjae Yu, Jae Sung Park, Jize Cao, Ali Farhadi, Yejin Choi
NeurIPS 2021 Oral (1%)

MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers
Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Yejin Choi, Zaid Harchaoui
NeurIPS 2021 Oral (1%) -- Outstanding Paper Award

Divergence Frontiers for Generative Models: Sample Complexity, Quantization Level, and Frontier Integral
Lang Liu, Krishna Pillutla, Sean Welleck, Sewoong Oh, Yejin Choi, Zaid Harchaoui
NeurIPS 2021

NaturalProofs: Mathematical Theorem Proving in Natural Language
Sean Welleck, Jiacheng Liu, Ronan Le Bras, Hannaneh Hajishirzi, Yejin Choi, Kyunghyun Cho
NeurIPS 2021 Datasets and Benchmarks Track Oral (1%)

CommonsenseQA 2.0: Exposing the Limits of AI through Gamification
Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, Jonathan Berant
NeurIPS 2021 Datasets and Benchmarks Track

CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras and Yejin Choi
EMNLP 2021

Moral Stories: Learning to Reason about Norms, Intents, Actions, and their Consequences from Short Narratives
Denis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes and Yejin Choi
EMNLP 2021

Conversational Multi-hop Reasoning with Neural Commonsense Knowledge and Symbolic Logic Rules
Forough Arabshahi, Jennifer Lee, Antoine Bosselut, Yejin Choi and Tom Mitchell
EMNLP 2021

Contrastive Explanations for Model Interpretability
Alon Jacovi, Swabha Swayamdipta, Shauli Ravfogel, Yanai Elazar, Yejin Choi and Yoav Goldberg
EMNLP 2021

Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right
Ari Holtzman*, Peter West*, Vered Shwartz, Yejin Choi and Luke Zettlemoyer
EMNLP 2021

proScript: Partially Ordered Scripts Generation
Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark and Yejin Choi
EMNLP 2021 Findings

Analyzing Commonsense Emergence in Few-shot Knowledge Models
Jeff Da, Ronan Le Bras, Ximing Lu, Yejin Choi, Antoine Bosselut
AKBC 2021

DExperts: On-the-Fly Controlled Text Generation with Experts and Anti-Experts
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah Smith and Yejin Choi
ACL 2021

PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World
Rowan Zellers, Ari Holtzman, Matthew Peters, Roozbeh Mottaghi, Aniruddha Kembhavi, Ali Farhadi and Yejin Choi
ACL 2021

TIMEDIAL: Temporal Commonsense Reasoning in Dialog
Lianhui Qin, Aditya Gupta, Shyam Upadhyay, Luheng He, Yejin Choi and Manaal Faruqui
ACL 2021

Reflective Decoding: Beyond Unidirectional Generation with Off-the-shelf Language Models
Peter West, Ximing Lu, Ari Holtzman, Chandra Bhagavatula, Jena D. Hwang and Yejin Choi
ACL 2021

Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation
Jeff Da, Maxwell Forbes, Rowan Zellers, Anthony Zheng, Jena D. Hwang, Antoine Bosselut and Yejin Choi
ACL 2021

On-the-Fly Attention Modulation for Neural Generation
Yue Dong, Chandra Bhagavatula, Ximing Lu, Jena D. Hwang, Antoine Bosselut, Jackie Chi Kit Cheung and Yejin Choi
ACL 2021 Findings

Go Figure: A Meta Evaluation of Factuality in Summarization
Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi and Jianfeng Gao
ACL 2021 Findings

VinVL: Making Visual Representations Matter in Vision-Language Models
Pengchuan Zhang*, Xiujun Li*, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, Jianfeng Gao
CVPR 2021

TuringAdvice: A Generative and Dynamic Evaluation of Language Use
Rowan Zellers, Ari Holtzman, Elizabeth Clark, Lianhui Qin, Ali Farhadi, Yejin Choi
NAACL 2021
*Project Page

NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
NAACL 2021

"I'm Not Mad": Commonsense Implications of Negation and Contradiction
Liwei Jiang, Antoine Bosselut, Chandra Bhagavatula, Yejin Choi
NAACL 2021

Discourse Understanding and Factual Consistency in Abstractive Summarization
Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, Yejin Choi
EACL 2021

Challenges in Automated Debiasing for Toxic Language Detection
Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi
EACL 2021

(Comet-) Atomic 2020: On Symbolic and Neural Commonsense Knowledge Graphs
Jena Hwang*, Chandra Bhagavatula*, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, Yejin Choi
AAAI 2021

🦄 Unicorn on Rainbow: A Universal Commonsense Reasoning Model on a New Multitask Benchmark
Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
AAAI 2021

Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
Antoine Bosselut, Ronan Le Bras, Yejin Choi
AAAI 2021

Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision
Faeze Brahman, Vered Shwartz, Rachel Rudinger, Yejin Choi
AAAI 2021

Paragraph-level Commonsense Transformers with Recurrent Memory
Saadia Gabriel, Chandra Bhagavatula, Vered Shwartz, Ronan Le Bras, Maxwell Forbes, Yejin Choi
AAAI 2021

MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversations
Yao Dou, Maxwell Forbes, Ari Holtzman, Yejin Choi
AAAI 2021

Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes
Nicholas Lourie, Ronan Le Bras, Yejin Choi
AAAI 2021

🔮 Social Chemistry 101: Learning to Reason about Social and Moral Norms
Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap and Yejin Choi
EMNLP 2020

Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning (DeLorean)
Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut and Yejin Choi
EMNLP 2020

PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction
Xinyao Ma, Maarten Sap, Hannah Rashkin and Yejin Choi
EMNLP 2020

Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith and Yejin Choi
EMNLP 2020

PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking
Hannah Rashkin, Asli Celikyilmaz, Yejin Choi and Jianfeng Gao
EMNLP 2020

Unsupervised Commonsense Question Answering with Self-Talk
Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
EMNLP 2020

Thinking Like a Skeptic: Defeasible Inference in Natural Language
Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith and Yejin Choi
EMNLP 2020 Findings

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi and Noah A. Smith
EMNLP 2020 Findings
*Demo

Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs
Ana Marasovic, Chandra Bhagavatula, Jae sung Park, Ronan Le Bras, Noah A. Smith and Yejin Choi
EMNLP 2020 Findings

CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, Xiang Ren
EMNLP 2020 Findings
*Project Page

Generative Data Augmentation for Commonsense Reasoning
Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi and Doug Downey
EMNLP 2020 Findings

Visual Comet: Reasoning about the Dynamic Context of a Still Image
Jae Sung Park, Chandra Bhagavatula, Roozbeh Mottaghi, Ali Farhadi, Yejin Choi
ECCV 2020
*Project Page

Oscar: Object-Semantics Aligned Pre-training for Vision-and-Language Tasks
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao
ECCV 2020

Adversarial Filters of Dataset Biases
Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew E. Peters, Ashish Sabharwal, Yejin Choi
ICML 2020

Social Bias Frames: Reasoning about Social and Power Implications of Language
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith & Yejin Choi
ACL 2020
*Project Page

Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models
Maarten Sap, Eric Horvitz, Yejin Choi, Noah A Smith & James W Pennebaker
ACL 2020
*Project Page

Procedural Reading Comprehension with Attribute-Aware Context Flow
Aida Amini, Antoine Bosselut, Bhavana Dalvi Mishra, Yejin Choi, Hannaneh Hajishirzi
AKBC, 2020 -- best paper nomination

The Curious Case of Neural Text Degeneration
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi
ICLR 2020

Abductive Commonsense Reasoning
Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih and Yejin Choi
ICLR 2020

WinoGrande: An Adversarial Winograd Schema Challenge at Scale
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
AAAI 2020 -- Outstanding Paper Award (given to the single best paper)
*Leaderboard, Annoucement

PIQA: Reasoning about Physical Commonsense in Natural Language
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, Yejin Choi
AAAI 2020
*Leaderboard

Commonsense Knowledge Base Completion with Structural and Semantic Context
Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, Yejin Choi
AAAI 2020

Do Neural Language Models Overcome Reporting Bias?
Vered Shwartz and Yejin Choi
COLING 2020

Defending Against Neural Fake News (Grover)
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, Yejin Choi
NeurIPS 2019
*Blog! (with additional experiments & analysis)
*Grover Demo!
*Project Page

BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
Peter West, Ari Holtzman, Jan Buys and Yejin Choi
EMNLP 2019
*Project Page

Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
Lifu Huang, Ronan Le Bras, Chandra Bhagavatula and Yejin Choi
EMNLP 2019
*Project Page & Leaderboard

Counterfactual Story Reasoning and Generation
Lianhui Qin, Antoine Bosselut, Ari Holtzman, Chandra Bhagavatula, Elizabeth Clark and Yejin Choi
EMNLP 2019
*Code & Data

Social IQa: Commonsense Reasoning about Social Interactions
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras and Yejin Choi
EMNLP 2019
*Project Page & Leaderboard

Robust Navigation with Language Pretraining and Stochastic Sampling
Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah A. Smith and Yejin Choi
EMNLP 2019

Early Fusion for Goal Directed Robotic Vision
Aaron Walsman, Yonatan Bisk, Saadia Gabriel, Dipendra Misra, Yoav Artzi, Yejin Choi, Dieter Fox
IROS 2019 -- best paper nomination

Do Neural Language Representations Learn Physical Commonsense?
Maxwell Forbes, Ari Holtzman, Yejin Choi
CogSci 2019
*Project Page

COMET: Commonsense Transformers for Knowledge Graph Construction
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz and Yejin Choi
ACL 2019
*Demo

Hellaswag: Can a Machine Really Finish Your Sentence?
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi and Yejin Choi
ACL 2019
*Project Page & Leaderboard
Featured in ZdNet: "No, This AI can't finish your sentence"

The Risk of Racial Bias in Hate Speech Detection
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi and Noah A. Smith
ACL 2019 -- best paper nomination

Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi and Jianfeng Gao
ACL 2019
*Code & Data

From Recognition to Cognition: Visual Commonsense Reasoning (VCR)
Rowan Zellers, Yonatan Bisk, Ali Farhadi, Yejin Choi.
CVPR 2019
*Project Page & Leaderboard

Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
Liyiming Ke, Xiujun Li, Yonatan Bisk, Ari Holtzman, Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi and Siddhartha Srinivasa
CVPR 2019
*Video Demo

ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning
Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith and Yejin Choi
AAAI 2019
*Demo at Ai2, *Project Page at UW

MathQA Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi and Hannaneh Hajishirzi.
NAACL 2019
*Project Page & Leaderboard

Benchmarking Hierarchical Script Knowledge
Yonatan Bisk, Jan Buys, Karl Pichotta and Yejin Choi.
NAACL 2019

DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension
Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, Claire Cardie
TACL 2019
*Leaderboard

SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Rowan Zellers, Yonatan Bisk, Roy Schwartz and Yejin Choi
EMNLP 2018
*Project Page *Leaderboard

QuAC: Question Answering in Context
Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang and Luke Zettlemoyer
EMNLP 2018
*Leaderboard

Neural Metaphor Detection in Context
Ge Gao, Eunsol Choi, Yejin Choi and Luke Zettlemoyer
EMNLP 2018 short paper

Modeling Naive Psychology of Characters in Simple Commonsense Stories
Hannah Rashkin, Antoine Bosselut, Maarten Sap, Kevin Knight and Yejin Choi
ACL 2018
*Project Page (with an online dataset browser!)

Learning to Write with Cooperative Discriminators
Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub and Yejin Choi
ACL 2018

Event2Mind: Commonsense Inference on Events, Intents, and Reactions
Hannah Rashkin, Maarten Sap, Emily Allaway, Noah A. Smith and Yejin Choi
ACL 2018
*Project Page (with an online dataset browser!)

Ultra-Fine Entity Typing
Eunsol Choi, Omer Levy, Yejin Choi and Luke Zettlemoyer
ACL 2018
*Project Page

Sounding Board: A User-Centric and Content-Driven Social Chatbot
Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A. Smith, and Mari Ostendorf
NAACL 2018 demo paper

Discourse-Aware Neural Rewards for Coherent Text Generation
Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-sen Huang, and Yejin Choi
NAACL 2018

Deep Communicating Agents for Abstractive Summarization
Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi
NAACL 2018

Neural Poetry Translation
Marjan Ghazvininejad, Yejin Choi, and Kevin Knight
NAACL 2018 short paper

Neural Motifs: Scene Graph Parsing with Global Context
Rowan Zellers, Mark Yatskar, Sam Thomson, and Yejin Choi
CVPR 2018

Simulating Action Dynamics with Neural Process Networks
Antoine Bosselut, Omer Levy, Ari Holtzman, Corin Ennis, Dieter Fox, and Yejin Choi
ICLR 2018
[Podcast]

Learning Interpretable Spatial Operations in a Rich 3D Blocks World
Yonatan Bisk, Kevin J. Shih, Yejin Choi, and Daniel Marcu
AAAI 2018

Bridging HMMs and RNNs through Architectural Transformations
Jan Buys, Yonatan Bisk, Yejin Choi
IRASL workshop @ NeurIPS 2018

Zero-Shot Activity Recognition with Verb Attribute Induction
Rowan Zellers and Yejin Choi
EMNLP 2017
*Project Page

Dynamic Entity Representations in Neural Language Models
Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi and Noah A. Smith
EMNLP 2017

Truth of Varying Shades: On Political Fact-Checking and Fake News
Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova and Yejin Choi
EMNLP 2017 short paper
*Project Page

Connotation Frames of Agency and Power in Modern Films
Maarten Sap, Marcella Cindy Prasettio, Ariel Holtzman, Hannah Rashkin and Yejin Choi
EMNLP 2017 short paper
*Project Page     *Demo Page

Neural AMR: Sequence-to-Sequence Models for Parsing and Generation
Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi and Luke Zettlemoyer
ACL 2017
*Project Page

Verb Physics: Relative Physical Knowledge of Actions and Objects
Maxwell Forbes and Yejin Choi
ACL 2017
*Project Page

Multilingual Connotation Frames: A Case Study on Social Media for Targeted Sentiment Analysis and Forecast
Hannah Rashkin, Eric Bell, Yejin Choi, and Svitlana Volkova
ACL 2017 short paper
*Project Page

The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
Roy Schwartz, Maarten Sap, Yannis Konstas, Li Zilles, Yejin Choi and Noah A. Smith
CoNLL 2017

Story Cloze Task: UW NLP System
Roy Schwartz, Maarten Sap, Yannis Konstas, Li Zilles, Yejin Choi and Noah A. Smith
LSDSem 2017 shared task (Best performing system), 2017.

Globally Coherent Text Generation with Neural Checklist Models
Chloé Kiddon, Luke Zettlemoyer, and Yejin Choi
EMNLP 2016

Generating Topical Poetry
Marjan Ghazvininejad, Xing Shi, Yejin Choi, and Kevin Knight
EMNLP 2016

Connotation Frames: A Data-Driven Investigation
Hannah Rashkin, Sameer Singh, and Yejin Choi
ACL 2016
*Project Page

Learning Prototypical Event Structure from Photo Albums
Antoine Bosselut, Jianfu Chen, David Warren, Hannaneh Hajishirzi, and Yejin Choi
ACL 2016
*Project Page

Document-level Sentiment Inference with Social, Faction, and Discourse Context
Eunsol Choi, Hannah Rashkin, Luke Zettlemoyer, and Yejin Choi
ACL 2016
*Project Page

Are Elephants Bigger than Butterflies? Reasoning about Sizes of Objects
Hessam Bagherinezhad, Hannaneh Hajishirzi, Yejin Choi, and Ali Farhadi
AAAI 2016

Selected Publications (2011 - 2015):

Segment-Phrase Table for Semantic Segmentation, Visual Entailment and Paraphrasing
Hamid Izadinia, Fereshteh Sadeghi, Santosh Kumar Divvala, Hannaneh Hajishirzi, Yejin Choi, and Ali Farhadi
International Conference on Computer Vision (ICCV), 2015. (oral)

Mise en Place: Unsupervised Interpretation of Instructional Recipes
Chloé Kiddon, Ganesa Thandavam Ponnuraj, Luke Zettlemoyer, and Yejin Choi
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2015.

TreeTalk: Composition and Compression of Trees for Image Descriptions
Polina Kuznetsova, Vicente Ordonez, Tamara Berg, and Yejin Choi.
Transaction of Association for Computational Linguistics (TACL), 2014. (presented at EMNLP 2014)

From Large Scale Image Categorization to Entry-Level Categories.
Vicente Ordonez, Jia Deng, Yejin Choi, Alexander C Berg, and Tamara L Berg
International Conference on Computer Vision (ICCV), 2013. -- Marr Prize (best paper award)

Connotation Lexicon: A Dash of Sentiment Beneath the Surface Meaning.
Song Feng, Jun Seok Kang, Polina Kuznetsova, and Yejin Choi.
Association for Computational Linguistics (ACL), 2013.
*Project Page        *Data: Connotation lexicon
Featured in Fast Company

BabyTalk: Understanding and generating simple image descriptions
Girish Kulkarni, Visruth Premraj, Vicente Ordonez, Sagnik Dhar, Siming Li, Yejin Choi, Alexander C Berg, Tamara L Berg
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011. -- Longuet Higgins Prize (test of time award) at CVPR 2021

Finding Deceptive Opinion Spam by Any Stretch of the Imagination.
Myle Ott, Yejin Choi, Claire Cardie, and Jeffrey Hancock.
Association for Computational Linguistics (ACL), 2011. -- Test of Time Award at ACL 2021
Featured in WNBC News for New York (Sep 2012, 5pm news); NHPR Radio (Sep 22, 2011); Bloomberg Business Week (Oct 2011); NY Times (Aug 19, 2011);

Full List of Publications:

Google Scholar
Semantic Scholar

Recent Teaching:

CS   224n   Natural Language Processing with Deep Learning [Winter 2026]
CSE   517 + 447   (Grad + Undergrad) Natural Language Processing --- an LLM edition [Winter 2024]
CSE   599 D1   (Grad) Exploration on Language, Knowledge, and Reasoning [Winter 2023]
CSEP 517   (Professional MS) Natural Language Processing [Winter 2021]
CSE   447   (Undergrad) Natural Language Processing [Winter 2020]
CSE   517   (Grad) Natural Language Processing [Winter 2019]
CSE   481 N   (Undergrad Capstone) Natural Language Processing [Spring 2018]
CSE   447   (Undergrad) Natural Language Processing [Winter 2018]
CSE   481 N   (Undergrad Capstone) Natural Language Processing [Spring 2017]
CSE   517   (Grad) Natural Language Processing [Winter 2017]
CSE   490 U (Undergrad) Natural Language Processing [Spring 2016]
CSEP 517   (Professional MS) Natural Language Processing [Autumn 2015]
CSE   599   (Grad) Advanced NLP [Spring 2015]
CSE   517   (Grad) Natural Language Processing [Winter 2015]

xlab:

• Current members:
  
  Post Doc:
         Yining Hong - co-advised with Leonidas Guibas
  
  PhD:
         Linjie (Lidsey) Li - co-advised with Ranjay Krishna
         Ben Newman
         Fang Wu- co-advised with Jure Leskovec
         Niklas Muennighoff- co-advised with Andrew Ng
         Konwoo Kim- co-advised with Nick Haber
         Matthew Liu - co-advised with Brian Hie
         Shiye Su - co-advised with Serena Yeung-Levy
  
  Undergrad:
  
  
• Former members:
  
  Research Scientist:
         Sydney Levine (-> Assistant Professor @ NYU after a gap year at Google DeepMind)
  Post Doc:
         Pan Lu (-> a stealth start-up)
         Niloofar Mireshghallah (-> Assistant Professor at CMU LTI/EPP after a gap year at Meta and Humans&)
         Chan Young Park (-> Assistant Professor at UT Austin ISchool after a gap year at MSR)
         Abhilasha Ravichander (->Assistant Professor at MPI-SWS)
         Hyunwoo Kim (->Assistant Professor at KAIST after a gap year at NVIDIA)
         Valentina Pyatkin (Young Investigator @ Ai2 -> ETH AI Center)
         Faeze Brahman (Research Scientist @ Ai2)
         Tianxiao Shen (-> Research Scientist at Google DeepMind)
         Yuntian Deng (-> Assistant Professor at Waterloo)
         Mitchell L Gordon (-> Assistant Professor at MIT)
         Sean Welleck (-> Assistant Professor at CMU)
         Nouha Dziri (-> Research Scientist @ Ai2 -> Cohere)
         Yuchen (Bill) Lin (-> Research Scientist @ Ai2 -> XAI)
         Prithviraj Ammanabrolu (-> Assistant Professor at UCSD after a gap year at MosaicML)
         Alane Suhr (-> Assistant Professor @ UC Berkeley)
         Xiang Lorraine Li (-> Assistant Professor @ University of Pittsburgh)
         Youngjae Yu (-> Assistant Professor @ Yonsei University -> SNU)
         Daniel Khashabi (-> Assistant Professor @ JHU)
         Swabha Swayamdipta (-> Assistant Professor @ USC)
         Vered Shwartz (-> Assistant Professor @ UBC)
         Jack Hessel (-> Research Scientist @ Ai2 -> Samaya AI -> Anthropic)
         Rachel Rudinger (-> Assistant Professor @ University of Maryland)
         Yonatan Bisk (-> Assistant Professor @ CMU LTI/Robotics)
         Jan Buys (-> Assistant Professor @ University of Cape Town)
         Yannis Konstas (-> Assistant Professor @ Heriot-Watt University)
  
  PhD:
         Seungju Han (-> Thinking Machines Lab)
         Liwei Jiang (-> Assistant Professor at UC Irvine after a gap year at NVIDIA)
         Alisa Liu (-> OpenAI)
         Melanie Sclar (-> Meta)
         Jaehun Jung (-> a stealth start-up)
         Ximing (Gloria) Lu (-> a stealth start-up)
         Etash Guha (-> Anthropic)
         Taylor Sorensen (-> Anthropic)
         Anas Awadalla (-> Anthropic)
         Jillian Fisher (-> Postdoc at Stanford)
         Jiacheng (Gary) Liu (-> Research Scientist @ XAI)
         Jae Sung (James) Park (-> Research Scientist @ Ai2)
         Peter West (-> Assistant Professor at UBC after a gap year at Stanford)
         Xiujun Li (-> Research Scientist @ Apple)
         Lianhui (Karen) Qin (-> Assistant Professor at UCSD)
         Saadia Gabriel (->Assistant Professor @ UCLA after a gap year at MIT and NYU)
         Maarten Sap (-> Assistant Professor at CMU)
         Rowan Zellers (-> OpenAI)
         Maxwell Forbes (-> Wandering around the world -> Anthropic)
         Antoine Bosselut (-> Assistant Professor @ EPFL)
         Hannah Rashkin (-> Research Scientist @ Google Research)
         Eunsol Choi (-> Assistant Professor @ Texas Austin -> NYU)
         Chloé Kiddon (-> Google)
         Jun Seok Kang (-> Blink Health)
         Song Feng (-> Research Staff Member @ IBM Research)
         Polina Kuznetsova (-> Research Scientist @ Facebook)
         Ritwik Banerjee (-> Research Assistant Professor @ Stony Brook University)
  
  Undergrad:
         Kavel Rao
         Jenny Liang (-> PYI @ Ai2 -> PhD @ CMU)
         Jeff Da (-> PYI @ Ai2)
         Yao Dou (-> PhD @ GaTech)
         Xinyao (Michelle) Ma (-> Pinterest)
         Ge Gao (-> PhD @ Cornell)
         Emily Louise Allaway (-> PhD @ Columbia -> Assistant Professor @ University of Edinburgh)
         Marcella Cindy Prasetio (-> MS @ Stanford)
         Ryan Benmalek (-> PhD @ Cornell)
         Pooja Sethi (-> Facebook)
  
  

Short Bio:

Yejin Choi is the Dieter Schwarz Foundation Professor of Computer Science at Stanford University and a Senior Fellow at the Stanford Institute for Human-Centered AI (HAI). She is a MacArthur Fellow and AI2050 Senior Fellow, and was named to the TIME100 Most Influential People in AI in 2023 and 2025. Her work has earned 2 Test-of-Time Awards and 12 Best/Outstanding Paper Awards at top AI venues including ACL, EMNLP, NAACL, ICML, NeurIPS, ICCV, CVPR, and AAAI. Beyond a main-stage talk at TED 2023, she has delivered keynotes at NeurIPS, ICLR, CVPR, ACL, AAAI, MLSys, VLDB, and WebConf, and has received the Borg Early Career Award (BECA, 2018), the inaugural Alexa Prize (2017), and IEEE AI’s 10 to Watch (2016). Her research centers on democratizing generative AI — building smaller yet powerful language models, scaling intelligence through smarter algorithms and alternative training recipes, pluralistic alignment, and AI for science and social good. She was previously a Senior Research Director and Distinguished Research Scientist at NVIDIA, a Professor at the University of Washington, and a Senior Director at Ai2. She holds a Ph.D. in Computer Science from Cornell University and a B.S. in Computer Engineering from Seoul National University.

Personal:

Scuba!