GeezerVisonThe World Wide Web, restored to 800 pixels of glory. |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites |
GeezerVison is showing valentinapy.github.io — Valentina Pyatkin I am a Researcher at the ETH AI Center, where I work on post-training of LLMs for the Swiss AI Initiative, contributing to Apertus. Previously, I was a postdoctoral researcher (and Young Investigator) at the Allen Institute for AI and the University of Washington, advised by Prof. Hanna Hajishirzi and Prof. Yejin Choi. There I worked on open post-training efforts such as Tülu 3 and contributed to OLMo, work recognized with the ACL Best Theme Paper Award. I completed my PhD in Computer Science at the NLP lab of Bar Ilan University, supervised by Prof. Ido Dagan and Prof. Reut Tsarfaty, during which I was fortunate to spend time as a visiting student at UW NLP with Prof. Yejin Choi and to intern twice at the Allen Institute for AI, where I received the Outstanding Intern of the Year Award. Earlier, I interned at Google Research, obtained an MSc in Artificial Intelligence with distinction from the University of Edinburgh, and a BA from the University of Zurich. My research interests center on post-training and adaptation of language models — making them precisely steerable. I care about how models follow instructions and constraints, and how preference and reward signals shape their behavior. This has led me to build benchmarks like IFBench and RewardBench, and to work broadly across data quality, evaluation methodology, and the science of LLM development. My work has been awarded an ACL Outstanding Paper Award and featured in the press, for example by TechCrunch and GeekWire. Feel free to get in touch! Work with me: Swiss AI InitiativeI am looking for motivated students to work with me on contributing to the Swiss AI Initiative’s LLM efforts, particularly with a focus on post-training. We are open to host students at ETHZ or EPFL for semester or thesis projects. Indicate interest using this application form, mentioning my name. ResearchMy research focuses on how one can develop generative AI that is contextually robust, responsible and open. In particular, I have focused on extending language models’ capabilities through post-training and adaptation. Additionally, I have been involved in the construction of multiple, widely-used benchmarks, such as RewardBench! More specifically, my research is centered around:
Awards
PublicationsBelow is a selection of my recent publications; for my full publication record, please see my Google Scholar page. 2026Apertus 1.5: From Open Text Models to Open Multimodal Reasoners Project Apertus, including Valentina Pyatkin (Core) XYBench: Can LLMs Respond Pragmatically to Queries with Misconceptions? Akhila Yerukola, Jena D Hwang, Mingqian Zheng, Jenna Godsey, Hyunwoo Kim, Valentina Pyatkin, Jennifer Hu, Maarten Sap EMNLP 2026 ★ Best Resource Award paper SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning Alexis Limozin, Eduard Durech, Torsten Hoefler, Imanol Schlag, Valentina Pyatkin NeurIPS 2026, Datasets and Benchmarks paper TurnWise: The Gap between Single- and Multi-turn Language Model Capabilities Victoria Graf, Valentina Pyatkin, Nouha Dziri, Nathan Lambert, Hannaneh Hajishirzi Which course? Discourse! Teaching Discourse and Generation in the Era of LLMs Junyi Jessy Li, Yang Janet Liu, Kanishka Misra, Valentina Pyatkin, William Sheffield TeachNLP @ EACL 2026 paper PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harm Jing-Jing Li, Joel Mire, Eve Fleisig, Valentina Pyatkin, Anne Collins, Maarten Sap, Sydney Levine ICLR 2026 paper RewardBench 2: Advancing Reward Model Evaluation Saumya Malik, Valentina Pyatkin, Sander Land, Jacob Morrison, Noah A. Smith, Hannaneh Hajishirzi, Nathan Lambert ICLR 2026 paper 2025IF-RLVR: Generalizing Verifiable Instruction Following Valentina Pyatkin, Saumya Malik, Victoria Graf, Hamish Ivison, Shengyi Huang, Pradeep Dasigi, Nathan Lambert, Hannaneh Hajishirzi NeurIPS 2025 paper TÜLU 3: Pushing Frontiers in Open Language Model Post-Training Valentina Pyatkin*, Nathan Lambert*, Jacob Morrison*, Shengyi Huang*, Hamish Ivison*, Faeze Brahman*, Lester James V. Miranda*, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, Hannaneh Hajishirzi COLM 2025 paper 2 OLMo 2 Furious Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, Hannaneh Hajishirzi COLM 2025 paper Diverging Preferences: When do Annotators Disagree and do Models Know? Michael J.Q. Zhang, Zhilin Wang, Jena D. Hwang, Yi Dong, Olivier Delalleau, Yejin Choi, Eunsol Choi, Xiang Ren, Valentina Pyatkin ICML 2025 paper SafetyAnalyst: Interpretable, transparent, and steerable LLM safety moderation Jing-Jing Li, Valentina Pyatkin, Max Kleiman-Weiner, Liwei Jiang, Nouha Dziri, Anne G. E. Collins, Jana Schaich Borg, Maarten Sap, Yejin Choi, Sydney Levine ICML 2025 paper RewardBench: Evaluating Reward Models for Language Modeling Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi NAACL Findings 2025 paper Superlatives in Context: Modeling the Implicit Semantics of Superlatives Valentina Pyatkin, Bonnie Webber, Ido Dagan, Reut Tsarfaty NAACL 2025 paper IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance Paul Röttger, Musashi Hinck, Valentin Hofmann, Kobi Hackenburg, Valentina Pyatkin, Faeze Brahman, Dirk Hovy TACL 2025 paper Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback Lester James V. Miranda*, Yizhong Wang*, Yanai Elazar, Sachin Kumar, Valentina Pyatkin, Faeze Brahman, Noah A. Smith, Hanna Hajishirzi, Pradeep Dasigi ACL 2025 paper WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, Yejin Choi ICLR 2025 paper 2024Explicating the Implicit: Argument Detection Beyond Sentence Boundaries Paul Roit, Aviv Slobodkin, Eran Hirsch, Arie Cattan, Ayal Klein, Valentina Pyatkin, Ido Dagan ACL 2024 paper Self-Directed Synthetic Dialogues and Revisions Technical Report Nathan Lambert, Hailey Schoelkopf, Aaron Gokaslan, Luca Soldaini, Valentina Pyatkin, Louis Castricato The Art of Saying No: Contextual Noncompliance in Language Models Faeze Brahman*, Sachin Kumar*, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, Yulia Tsvetkov, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi NeurIPS 2024 paper Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback Hamish Ivison, Yizhong Wang, Jiacheng Liu, Zeqiu Wu, Valentina Pyatkin, Nathan Lambert, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi NeurIPS 2024 paper Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models Paul Röttger*, Valentin Hofmann*, Valentina Pyatkin, Musashi Hinck, Hannah Rose Kirk, Hinrich Schütze, Dirk Hovy ACL 2024 ★ Outstanding Paper Award paper OLMo: Accelerating the Science of Language Models Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi ACL 2024 ★ Best Theme Paper Award paper Promptly Predicting Structures: The Return of Inference Maitrey Mehta, Valentina Pyatkin, Vivek Srikumar NAACL 2024 paper Retrieving Texts based on Abstract Descriptions Shauli Ravfogel, Valentina Pyatkin, Amir DN Cohen, Avshalom Manevich, Yoav Goldberg COLM 2024 paper 2023Camels in a Changing Climate: Enhancing LM Adaptation with TÜLU 2 Hamish Ivison*, Yizhong Wang*, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi "You Are An Expert Linguistic Annotator": Limits of LLMs as Analyzers of Abstract Meaning Representation Allyson Ettinger, Jena D. Hwang, Valentina Pyatkin, Chandra Bhagavatula, Yejin Choi EMNLP Findings 2023 paper What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations Kavel Rao, Liwei Jiang, Valentina Pyatkin, Yuling Gu, Niket Tandon, Nouha Dziri, Faeze Brahman, Yejin Choi EMNLP Findings 2023 paper Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, Xiang Ren ICLR 2024 paper Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties Taylor Sorensen, Liwei Jiang, Jena Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas, Yejin Choi AAAI 2024 paper PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, Jena D. Hwang, Xiang Lorraine Li, Hirona J. Arai, Soumya Sanyal, Keisuke Sakaguchi, Xiang Ren, Yejin Choi ICLR 2024 paper Design Choices for Crowdsourcing Implicit Discourse Relations: Revealing the Biases Introduced by Task Design Valentina Pyatkin, Frances Yung, Merel C.J. Scholman, Reut Tsarfaty, Ido Dagan, Vera Demberg TACL 2023 paper ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations Valentina Pyatkin, Jena D. Hwang, Vivek Srikumar, Ximing Lu, Liwei Jiang, Yejin Choi, Chandra Bhagavatula ACL 2023 paper Revisiting Sentence Union Generation as a Testbed for Text Consolidation Eran Hirsch, Valentina Pyatkin, Ruben Wolhandler, Avi Caciularu, Asi Shefer, Ido Dagan ACL Findings 2023 paper 2022Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE Yuling Gu, Yao Fu, Valentina Pyatkin, Ian H. Magnusson, Bhavana Dalvi, Peter Clark FigLang @ EMNLP 2022 paper QASem Parsing: Text-to-text Modeling of QA-based Semantics Ayal Klein, Eran Hirsch, Ron Eliav, Valentina Pyatkin, Avi Caciularu, Ido Dagan EMNLP 2022 paper Design Choices in Crowdsourcing Discourse Relation Annotations: The Effect of Worker Selection and Training Merel C.J. Scholman, Valentina Pyatkin, Frances Yung, Ido Dagan, Reut Tsarfaty, Vera Demberg LREC 2022 paper Draw Me a Flower: Grounding Formal Abstract Structures Stated in Informal Natural Language Royi Lachmy, Valentina Pyatkin, Avshalom Manevich, Reut Tsarfaty TACL 2022 paper 2021Asking It All: Generating Contextualized Questions for any Semantic Role Valentina Pyatkin*, Paul Roit*, Julian Michael, Reut Tsarfaty, Yoav Goldberg, Ido Dagan EMNLP 2021 paper The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing Valentina Pyatkin*, Shoval Sadde*, Aynat Rubinstein, Paul Portner, Reut Tsarfaty ACL 2021 paper 2020QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines Valentina Pyatkin, Ayal Klein, Reut Tsarfaty, Ido Dagan EMNLP 2020 paper QA-Nom: Question-Answer driven SRL for Nominalizations Ayal Klein, Jonathan Mamou, Valentina Pyatkin, Daniela Stepanov, Hangfeng He, Dan Roth, Luke Zettlemoyer, Ido Dagan COLING 2020 paper 2017Discourse Relations and Conjoined VPs: Automated Sense Recognition Valentina Pyatkin, Bonnie Webber EACL SRW 2017 paper * denotes equal contribution. Misc
Outside of work I love rowing, hiking, and going to the cinémathèque. I think Italian Neorealism produced some of the most beautiful films ever made, such as Rocco e i suoi fratelli. And I’m a Monica Vitti fan. My Erdős number is 3 (Paul Erdős → Noga Alon → Ido Dagan → me) and my Kevin Knight number is 2 (Kevin Knight → Yejin Choi → me). |
| You are visitor 01402503 — GeezerVison does not store the pages you look up. Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required. |