Nick Gregory is an AI researcher at XBOW, where he builds offensive security agents, and the co-founder and former chief technology officer of Bismuth. His work addresses a central weakness of autonomous software engineering: systems that generate convincing code and bug reports often struggle to identify defects that actually matter.
Gregory graduated from New York University’s Tandon School of Engineering in 2019 after studying computer science and helping lead OSIRIS Lab and its CSAW capture-the-flag competition. In 2017, he co-created and co-taught an offensive-security course. He subsequently worked on embedded-systems emulation at MIT Lincoln Laboratory, production engineering at Facebook, exploitation detection at Capsule8 and Sophos, and endpoint-security software at Google.
His independent research includes discovering CVE-2022-25636, a Linux-kernel vulnerability enabling privilege escalation and container escape; developing snapshot-based fuzzing techniques adopted by AFL++; and creating Warpspeed, a record-and-replay debugger for macOS, with Pete Markowsky.
In 2024, Gregory founded Bismuth with Ian Butler, initially combining simplified cloud deployment with AI-assisted development. The company subsequently concentrated on bug detection and code generation using code search, static analysis, fuzzing, and language-model reasoning.
- SM-100: Gregory and Butler developed a software-maintenance benchmark containing 100 validated bugs from more than 84 public repositories across Python, TypeScript, JavaScript, and Go. It measures bug discovery, pull-request review, repair, and false positives against actual software defects.
- False positives determine practical usefulness: Their AI Engineer World’s Fair 2025 findings included a basic agent loop with a 97 percent false-positive rate, demonstrating that raw report volume can overwhelm engineers without improving software reliability.
- Cross-file reasoning and program comprehension: Reliable agents must trace data flow and calling context across a codebase; an apparently unsafe function may be harmless when its callers enforce the necessary constraints.
