GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Rubric Robustness
  • SpreadsheetArena
  • ResearchRubrics
  • Perplexity AI
  • RLHF Book
  • Chain of Thought
  • MCP-Atlas
  • OpenAI
  • Anthropic
  • Remote Labor Index
  • Center for AI Safety
  • Adaptive Guidance
  • Scale AI
  • Why Would You Suggest That?

This view

  • Original page
  • New address

GeezerVison is showing manasi-sharma.github.io — Manasi Sharma

Manasi Sharma

Hi there! I'm a Research Engineer at Scale AI, where I work on evaluations, benchmarking, and reinforcement learning / post-training research, to train frontier models with better reasoning capabilities for state-of-the-art agents like browser & computer-use agents. My work has been published at conferences such as ICLR, ICML, NeurIPS, and CoRL. Previously, I was a Research Scientist at MIT Lincoln Laboratory. I earned my MS in Computer Science (AI track) from Stanford and my BA in Computer Science and Physics from Columbia. I'm a generalist passionate about applying AI to diverse real-world challenges, from astronomy to autonomous vehicles to agents.

I am fortunate to have worked with many different professors over the course of my academic career. I conducted research in the ILIAD Lab under Prof. Dorsa Sadigh at Stanford on robotic learning using language-conditioned diffusion models, and the Stanford Vision Lab under Prof. Fei-Fei Li and Prof. Jiajun Wu on the BEHAVIOR robotic simulation benchmark. As an undergrad, I worked with Prof. Daniel Hsu and Prof. Zoltan Haiman on interpretable deep learning for astrophysics, and at Caltech with Prof. Mansi Kasliwal on image classification for the Gattini-IR telescope.

My research interests include AI evaluation methodologies, agents, reinforcement learning, and AI safety and alignment.

Everything else: Outside of work, I love dancing (I was Captain of the Columbia Raas dance team), playing the keyboard and exploring the natural landscapes in the Bay Area!

Email / Google Scholar / LinkedIn / Twitter / CV / GitHub
Manasi Sharma

Updates

  • Apr 2026 Rubric Robustness and SpreadsheetArena accepted at ICML 2026
  • Jan 2026 ResearchRubrics, a benchmark to evaluate deep research agents, was accepted at ICLR 2026
    It was also referenced in a Perplexity AI deep research release and in the RLHF Book
  • Jan 2026 I recorded a podcast! Check out the Chain of Thought podcast episode on ResearchRubrics
  • Jan 2026 MCP-Atlas preprint released and was referenced in OpenAI & Anthropic model card releases
  • Oct 2025 Remote Labor Index preprint released, in collaboration with the Center for AI Safety at UC Berkeley
  • Oct 2025 Adaptive Guidance accepted at the NeurIPS 1st Efficient Reasoning Workshop
  • Mar 2025 Joined Scale AI as ML Research Engineer on the Reasoning & Agents Research Team!
  • Jun 2024 Why Would You Suggest That? presented at ICML Workshop
  • Dec 2023 Exploring the Spatial Reasoning Abilities of LLMs presented at NeurIPS Instruction Tuning Workshop
  • Sep 2023 My Python tutorial for Stanford's CS 224N course was released on YouTube
  • Sep 2022 BEHAVIOR-1K accepted at CoRL 2022 and nominated for Best Paper

Education

Stanford University

Stanford University, School of Engineering

Master's in Computer Science

Advisors: Dorsa Sadigh (ILIAD Lab), Fei-Fei Li, Jiajun Wu (Stanford Vision Lab)

Columbia University

Columbia University, Columbia College

BA in Computer Science with concentration in Physics

Advisors: Daniel Hsu, Zoltan Haiman

Work Experience

Scale AI

Scale AI

Mar 2025 — Present
ML Research Engineer, Reasoning & Agents Research Team, Scale Labs — San Francisco, CA
  • Research Lead for computer-use & browser-use agents, spearheading 3 first-author publications (7 total) on evaluations at top-tier conferences (ICLR, ICML '26); including ResearchRubrics, referenced in a Perplexity AI agent release.
  • Collaborated with external organizations like the Center for AI Safety at Berkeley on benchmarks for economically useful tasks (Remote Labor Index) and tool-use (MCP-Atlas), referenced in OpenAI & Anthropic's latest model cards.
  • Post-trained models with RL to analyze strategies for optimal data learning and contributed to research on an efficient RL algorithm with intermediate rewards (NeurIPS '25 workshop).
  • Collaborated with researchers at top frontier labs, offering consultation on advanced computer-use data types involving programmatic & agentic verifiers.
MIT Lincoln Laboratory

MIT Lincoln Laboratory

Aug 2023 — Mar 2025
Research Scientist, Artificial Intelligence Technology Group — San Francisco, CA (Remote)
  • Spearheaded a project on LLM explainability & trust that was published at an ICML '24 workshop, & built an open-source LLM-testing framework (maite). Participated in a project collaboration with MIT Profs. Philip Isola & Jacob Andreas.
  • Architected an internal tool for efficient evaluation of a Retrieval Augmented Generation (RAG) system with 15+ distinct metrics & assessed it on referencing past recordings in a speech-to-text application, achieving 92% accuracy.
Nissan Research

Nissan Research

Jun 2022 — Sep 2022
Research Intern, Autonomous Systems, Alliance Innovation Lab — Santa Clara, CA
  • Engineered an end-to-end LiDAR 3D point-cloud classification system for autonomous vehicles, achieving >95% accuracy, ~2% FPR and 85% reduction in runtime. Deployed in Nissan Autonomous Vehicles beginning Winter '22.
Google

Google

Mar 2022 — Jun 2022
CS Research Mentorship Program — Mountain View, CA
  • Ideated research projects on language models & diffusion models and attended various mentorship events as part of Google's CSRMP.

Publications

2026
ICLR
ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents
Manasi Sharma, Chen Bo Calvin Zhang, Chaitanya Bandi, et al.
The Fourteenth International Conference on Learning Representations (ICLR), 2026
Paper Code Dataset Website
ICML
RubricRobustness: Evaluating the Sensitivity of Rubrics-Based Benchmarks to Simple Perturbations
Manasi Sharma, Brad Kenstler, Bing Liu
International Conference on Machine Learning (ICML), 2026 (arXiv coming soon)
Paper
Preprint
Agentic Rubrics for Test-Time Scaling of Computer-Use Agents
Manasi Sharma, Daniel Zhang, Bing Liu
Preprint, May 2026 (arXiv coming soon)
arXiv
MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers
Chaithanya Bandi, Ben Hertzberg, ..., Manasi Sharma, et al.
Preprint, January 2026
Paper Code Dataset Leaderboard
ICML
SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks
Srivatsa Kundurthy, Clara Na, Michael Handley, Zach Kirshner, Chen Bo Calvin Zhang, Manasi Sharma, Emma Strubell, John Ling
International Conference on Machine Learning (ICML), 2026
Paper Website
2025
arXiv
Remote Labor Index: Measuring AI Automation of Remote Work
Mantas Mazeika*, Alice Gatti*, Cristina Menghini*, Udari Madhushani Sehwag*, Shivam Singhal*, Yury Orlovskiy*, Steven Basart, Manasi Sharma, et al.
Preprint, October 2025
Paper Code Dataset Website
NeurIPS
Adaptive Guidance Accelerates Reinforcement Learning of Reasoning Models
Vaskar Nath, Elaine Lau, Anisha Gunjal, Manasi Sharma, Nikhil Baharte, Sean Hendryx
NeurIPS The First Workshop on Efficient Reasoning, 2025
Paper Code
2024
ICML
Why Would You Suggest That? Human Trust in Language Model Responses
Manasi Sharma, Ho Chit Siu, Rohan Paleja, Jaime D. Peña
ICML, Humans, Algorithmic Decision-Making and Society Workshop, 2024
Paper
2023
NeurIPS
Exploring and Improving the Spatial Reasoning Abilities of Large Language Models
Manasi Sharma
NeurIPS Instruction Tuning and Following Workshop, 2023
Paper
2022
CoRL
BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation
Chengshu Li, Cem Gokmen, ..., Manasi Sharma, ...
Conference on Robot Learning (CoRL), 2022 [Nominated for Best Paper]
Paper Code Website
2020
Phys Rev D
Interpreting deep learning models for weak lensing
Jose Manuel Zorrilla Matilla, Manasi Sharma, Daniel Hsu, Zoltan Haiman
Physical Review D, 102(12), American Physical Society, 2020
Paper
PASP
Palomar Gattini-IR: Survey Overview, Data Processing System, on-Sky Performance and First Results
Kishalay De, Matthew J. Hankins, ..., Manasi Sharma, ...
Publications of the Astronomical Society of the Pacific, vol. 132, 2020
Paper

Teaching Experience

Stanford Engineering

Stanford School of Engineering

Mar 2022 — Jun 2023

Graduate Teaching Assistant

  • Managed weekly Discussion Sections of 75+ students for some of the most popular CS courses at Stanford (>600 students); held office hours, constructed & graded HWs. Received >95% excellent reviews.
  • Courses: CS 224N (NLP, Prof. Manning), CS 231N (Computer Vision, Prof. Fei-Fei Li), CS 230 (Deep Learning, Prof. Andrew Ng)
  • Recorded the Python tutorial for Stanford's CS 224N course on YouTube
Columbia University

Columbia University, Department of Mathematics

Sep 2019 — Jun 2021

Undergraduate Teaching Assistant for Calculus III (across 4 semesters)

  • Graded assignments, led weekly office hours for ~150 students. Consistently received >80% excellent reviews.

Community Engagement

Stanford WiCS

Graduate Community Chair, Stanford Women in Computer Science

Jun 2022 — Jun 2023
  • Organized multiple events for the graduate community including mixers, industry alumni panels, lunches, talks, etc.; spearheaded a focus on MS students through the setup of an MS student alumni panel and care-package mixer.
  • Managed over $3000 in funds from the CS department and Engineering schools to fund events for graduate students.

Honors

  • 1 of 25 students accepted to the highly selective Pear Garage program for Entrepreneurship at Pear VC — Oct '22
  • 1 of 18 accepted to the highly selective GFSD (Graduate Fellowships for STEM Diversity) Program — Mar '22
  • 1 of 50 accepted into Google's CS Research Mentorship Program (CSRMP), Class of 2022A — Feb '22
  • Selected for the final round of the GEM Fellowship — Jan '22
  • Dean's List (in 6 out of 7 graded semesters, awarded to top 20%), Columbia University — Fall '17 - Fall '20
  • Columbia Undergraduate Research Fellowship (URF), Columbia College Summer Funding Program — May '20
  • Visiting Undergraduate Research Program (VURP) Award, California Institute of Technology — May '19
  • 1 of 25 awarded Laidlaw Undergraduate Research & Leadership Scholarship, Columbia University — '18 – '19
  • Andy Grove Scholarship for Intel Employees' Children, Intel Foundation — Fall '19

Areas of Experience

Deep Learning, Natural Language Processing, Generative AI, Agents, Reinforcement Learning & Decision Making, Computer Vision, Diffusion Models, Explainability, Robotics, & Graph Neural Networks.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.