Colin Flaherty is an AI researcher who contributed to Meta FAIR’s Cicero, which achieved human-level performance in the strategy game Diplomacy, and helped develop Augment Code’s open-source SWE-bench agent. His work addresses a central challenge in autonomous software engineering: equipping coding agents to understand unfamiliar environments, improve their own capabilities, and verify their output.
At Facebook AI Research, Flaherty coauthored the 2022 Science paper on Cicero, which combined natural-language interaction and strategic reasoning in a game requiring negotiation, cooperation, and long-term planning. He subsequently became a founding researcher at Augment Code, developing coding agents and retrieval systems.
With Tongfei Chen, he built an open-source SWE-bench Verified system that paired Claude Sonnet 3.7’s candidate fixes with OpenAI o1’s solution selection. The system reached 65.4% on the benchmark and led its open-source leaderboard at the time; Augment released the agent and evaluation infrastructure. By June 2025, Flaherty had left Augment and was working on a new, publicly unspecified project.
How he thinks about coding agents
- Self-coding agents need supervised feedback. An agent Flaherty helped develop extended its own codebase, implemented integrations, and wrote tests under human supervision. After adding Google Search, it located Linear’s unfamiliar API documentation and built another integration. It also instrumented itself, identified synchronous repository processing as a bottleneck, introduced parallel execution, and generated a stress test. His AI Engineer talk on self-coding agents explores how these capabilities compound.
- Organizational context determines practical autonomy. Agents need access to repositories, internal documentation, collaboration tools, execution environments, and human clarification. When an agent mishandled Graphite’s stacked-pull-request workflow, Flaherty’s team supplied searchable documentation explaining local commands and conventions. For SWE-bench repositories, iterative searches with grep and find proved more useful than assuming embedding-based retrieval would resolve the principal bottleneck.
- Executable tests make autonomy trustworthy. Flaherty identified a concurrency failure in which parallel agents could overwrite shared cache data despite apparently sensible locking. His response is to place executable tests inside the agent’s feedback loop so it can inspect failures and revise its work. As code becomes cheaper to generate, he expects reliable evaluation, product judgment, interaction design, and understanding users’ needs to become more consequential.
