Neel Guha
publications

See my Google Scholar for a complete list.

2026

  1. There’s No Free Benchmark: An Institutional View of Legal AI Benchmarking
    Neel Guha, Andy K. Zhang, Christine Tsang, Christopher D. Manning, Julian Nyarko, and Daniel E. Ho
    Proceedings of the National Academy of Sciences
  2. Cartridges: Lightweight and General-Purpose Long Context Representations via Self-Study
    Sabri Eyuboglu, Ryan Ehrlich, Simran Arora, Neel Guha, Dylan Zinsley, Emily Liu, Will Tennien, Atri Rudra, James Zou, Azalia Mirhoseini, and Christopher Ré
    International Conference on Learning Representations
    arXiv Code Blog
  3. AI Assurance
    Neel Guha and Izak C. Rosenfeld
    in Equalizing Justice: Harnessing AI for Litigants Without Lawyers, edited by Branting, L. Karl and Bonardi, Mia

2025

  1. A Reasoning-Focused Legal Retrieval Benchmark
    Lucia Zheng, Neel Guha, Javokhir Arifov, Sarah R Zhang, Michal Zhang, Michal Skreta, Christopher D Manning, Peter Henderson, and Daniel E Ho
    4th ACM Symposium on Computer Science and Law
    PDF Blog
  2. Archon: An Architecture Search Framework for Inference-Time Techniques
    Jon Saad-Falcon, Adrian Gamarra Lafuente, Shlok Natarajan, Nahum Maru, Hristo Todorov, Etash Guha, E. Kelly Buchanan, Mayee Chen, Neel Guha, Christopher Ré, and Azalia Mirhoseini
    International Conference on Machine Learning
    arXiv Code
  3. Open Problems in Technical AI Governance
    Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexandra Sasha Luccioni, Nitarshan Rajkumar, Nicolas Moës, Jeffrey Ladish, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland, Sanmi Koyejo, Mykel J. Kochenderfer, and Robert Trager
    Transactions on Machine Learning Research
    arXiv
  4. Building GenAI Benchmarks: A Case Study in Legal Applications
    Neel Guha, Julian Nyarko, Daniel E. Ho, and Christopher Ré
    in The Oxford Handbook on the Foundations and Regulation of Generative AI, edited by Hacker, Philipp and Engel, Andreas and Hammer, Sarah and Mittelstadt, Brent
    PDF

2024

  1. AI Regulation Has Its Own Alignment Problem: The Technical and Institutional Feasibility of Disclosure, Registration, Licensing, and Auditing
    Neel Guha, Christie M. Lawrence, Lindsey A. Gailmard, Kit T. Rodolfa, Faiz Surani, Rishi Bommasani, Inioluwa Deborah Raji, Mariano-Florentino Cuéllar, Colleen Honigsberg, Percy Liang, and Daniel E. Ho
    George Washington Law Review
    PDF
  2. Understanding Liability Risk from Using Health Care Artificial Intelligence Tools
    Michelle M. Mello and Neel Guha
    New England Journal of Medicine
    PDF DOI
  3. Stronger Than You Think: Benchmarking Weak Supervision on Realistic Tasks
    Tianyi Zhang, Linrong Cai, Jeffrey Li, Nicholas Roberts, Neel Guha, and Frederic Sala
    Advances in Neural Information Processing Systems (Datasets and Benchmarks Track)
    PDF Code Website
  4. The State Statutes Project
    Neel Guha and Diego Zambrano
    University of Wisconsin Law Review
    PDF
  5. Smoothie: Label Free Language Model Routing
    Neel Guha, Mayee F Chen, Trevor Chow, Ishan S Khare, and Christopher Ré
    Advances in Neural Information Processing Systems
    arXiv Code Blog
  6. Private Enforcement in the States
    Diego Zambrano, Neel Guha, Austin Peters, and Jeffrey Xia
    University of Pennsylvania Law Review
    PDF
  7. Prospector Heads: Generalized Feature Attribution for Large Models & Data
    Gautam Machiraju, Alexander Derry, Arjun Desai, Neel Guha, Amir-Hossein Karimi, James Zou, Russ Altman, Christopher Ré, and Parag Mallick
    International Conference on Machine Learning
    arXiv Code
  8. Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT
    Jon Saad-Falcon, Daniel Y. Fu, Simran Arora, Neel Guha, and Christopher Ré
    International Conference on Machine Learning
    arXiv Code Blog

2023

  1. LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
    Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Choi, Kevin Tobia, Margaret Hagan, Megan Ma, Michael Livermore, Nikon Rasumov-Rahe, Nils Holzenberger, Noam Kolt, Peter Henderson, Sean Rehaag, Sharad Goel, Shang Gao, Spencer Williams, Sunny Gandhi, Tom Zur, Varun Iyer, and Zehua Li
    Advances in Neural Information Processing Systems
    arXiv Code Website
  2. Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
    Neel Guha, Mayee F Chen, Kush Bhatia, Azalia Mirhoseini, Frederic Sala, and Christopher Ré
    Advances in Neural Information Processing Systems
    arXiv Code Blog
  3. Don’t Use a Cannon to Kill a Fly: An Efficient Cascading Pipeline for Long Documents
    Zehua Li, Neel Guha, and Julian Nyarko
    International Conference on AI and Law
    PDF
  4. ChatGPT and Physicians’ Malpractice Risk
    Michelle M. Mello and Neel Guha
    JAMA Health Forum
    PDF
  5. Gamesmanship in Modern Discovery Tech
    Diego Zambrano, Neel Guha, and Peter Henderson
    in Legal Tech and the Future of Civil Justice, edited by Engstrom, David Freeman
    PDF
  6. Holistic Evaluation of Language Models
    Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Andrew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda
    Transactions on Machine Learning Research
    arXiv Website
  7. Ask Me Anything: A Simple Strategy for Prompting Language Models
    Simran Arora, Avanika Narayan, Mayee F Chen, Laurel J Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala, and Christopher Ré
    International Conference on Learning Representations
    Spotlight
    arXiv Code

2022

  1. Vulnerabilities in Discovery Tech
    Neel Guha, Peter Henderson, and Diego Zambrano
    Harvard Journal of Law & Technology
    PDF
  2. LegalBench: Prototyping a Collaborative Benchmark for Legal Reasoning
    Neel Guha, Daniel E. Ho, Julian Nyarko, and Christopher Ré
    Preprint
    arXiv
  3. Pile of Law: Learning Responsible Data Filtering from the Law and a 256gb Open-source Legal Dataset
    Peter Henderson, Mark Krass, Lucia Zheng, Neel Guha, Christopher D Manning, Dan Jurafsky, and Daniel Ho
    Advances in Neural Information Processing Systems
    Oral Presentation
    arXiv Code Website

2021

  1. On the Opportunities and Risks of Foundation Models
    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang
    ArXiv
    arXiv
  2. When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset
    Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, and Daniel E. Ho
    International Conference on AI and Law
    Carole Hafner Award for Best Paper
    arXiv Code
  3. Leveraging Administrative Data for Bias Audits: Assessing Disparate Coverage with Mobility Data for COVID-19 Policy
    Amanda Coston, Neel Guha, Derek Ouyang, Lisa Lu, Alexandra Chouldechova, and Daniel E Ho
    ACM Conference on Fairness, Accountability, and Transparency
    arXiv Code
  4. Bootleg: Chasing the Tail with Self-Supervised Named Entity Disambiguation
    Laurel J. Orr, Megan Leszczynski, Simran Arora, Sen Wu, Neel Guha, Xiao Ling, and Christopher Ré
    Conference on Innovative Data Systems Research
    arXiv PDF Code Blog Website

2019

  1. Machine Learning for AC Optimal Power Flow
    Neel Guha, Zhecheng Wang, Matt Wytock, and Arun Majumdar
    Climate Change Workshop at the International Conference on Machine Learning
    Best Paper (Honorable Mention)
    arXiv
  2. One-Shot Federated Learning
    Neel Guha, Ameet Talwalkar, and Virginia Smith
    2nd Workshop on Machine Learning on the Phone and other Consumer Devices at Neural Information Processing Systems
    Oral Presentation
    arXiv
Powered by Jekyll. Website inspiration from Nicholas Carlini with assistance from Claude Code.