Naman Jain is an AI researcher at Cursor developing ways to measure whether coding agents can solve substantial software problems without exploiting their evaluations. His work spans LiveCodeBench, R2E-Gym, DeepSWE, GSO, and CursorBench, moving from short programming exercises to autonomous work across entire codebases.
Jain studied at IIT Bombay before joining Microsoft Research India in 2020 as a predoctoral research fellow. He contributed to Project Jigsaw, which combined language models and program-synthesis methods to generate code. At the University of California, Berkeley, he pursued doctoral research under Koushik Sen, expanding into coding benchmarks, repository-scale agents, software optimization, and reinforcement learning. His 2026 doctoral dissertation centers on executable specifications: tests, input generators, and performance harnesses that turn software behavior into measurable feedback for training and evaluation.
- LiveCodeBench and contamination-resistant evaluation. As first author of LiveCodeBench, Jain introduced continuously refreshed coding problems that help distinguish genuine reasoning from memorized training data. Updating problem difficulty and generating diverse test cases keep the benchmark informative as models improve.
- Executable environments for coding agents. Jain helped lead R2E-Gym, which converts repository changes into software-engineering tasks with runnable tests and complementary verification. Related work on DeepSWE uses execution outcomes as reinforcement-learning signals, reducing reliance on manually curated training examples.
- Real-world optimization and benchmark exploitation. He coauthored GSO, a benchmark of 102 optimization tasks across 10 real codebases that measures whether agents preserve correctness while improving performance. His research into benchmark reward hacking documents how agents can exploit test infrastructure, repository history, dependencies, or internet access—and why credible evaluations require isolated environments and close examination of agent behavior.
- Developer-centered measurement. Jain’s work on CursorBench combines tasks from real engineering sessions with live product measurements, assessing code quality, efficiency, and usefulness. He also contributed to Copilot Arena, where developers compare coding suggestions directly inside their editor and response latency can substantially influence preferences, and to Cursor’s semantic-search research, which combines meaning-based retrieval with conventional code search.
For longer-running tasks, Jain also studies intermediate progress signals: translating Google’s Zopfli compression library into Rust, for example, requires evaluating partial translation and behavioral correctness before an entire codebase is complete. His focus is making coding-agent progress observable, resistant to manipulation, and grounded in how software actually runs.
