Lawrence Jones is a founding engineer at incident.io who leads engineering on its AI SRE work, developing systems that investigate production failures across logs, metrics, traces, incident histories, and source code. His central challenge is making automated incident investigation reliable without allowing the agents responsible for it to become impossible to debug.
Jones joined GoCardless in 2015 and became a principal engineer on its site reliability team, building self-service production infrastructure and working on an Open Banking payments product. He joined incident.io in 2021, bringing operational experience to a young company focused on incident response.
He introduced the Goa framework into incident.io’s API development, using shared specifications to generate type-safe implementations, client libraries, documentation, and OpenAPI definitions. His subsequent work included Status Pages and Catalog, with the catalog informed by his experience maintaining a service registry at GoCardless. He designed for flexible schemas, incremental adoption, and actionable ownership information: a catalog becomes useful when it helps responders identify the right service or team during an incident. His account of building the catalog rejects elaborate registries that demand constant maintenance without improving operational decisions.
Jones also addressed a particularly unforgiving reliability problem: how an incident-management platform alerts its own engineers when the platform itself fails. His approach uses independent backup alerting, maintained separately and exercised against incident.io’s own alerts to preserve an emergency notification path during outages. He details the architecture in his explanation of resilient internal paging.
Making investigation agents testable
Jones applies established reliability-engineering practices to AI systems whose diagnoses can emerge from complicated chains of prompts, tools, and specialist agents. His AI Engineer Europe talk outlines three practical techniques:
- Prompt evaluations as unit tests: YAML test cases live beside Go code, with explicit grading criteria and regression checks. An agent-friendly Eval Tool lets coding agents inspect and update individual cases without loading oversized production-derived datasets into context, then reproduce failures, modify prompts, rerun evaluations, and simplify the result.
- Agent-readable debugging filesystems: Investigation traces, prompt hierarchies, inputs, and debugging views are exported into structured files that Claude Code or Codex can search alongside the codebase. Agents can locate the source of an incorrect diagnosis, propose a targeted fix, and validate it against existing evaluations.
- Failure-focused investigation backtesting: Parallel agents analyze individual investigations, cluster recurring failure patterns, and connect their findings to specific prompts or implementation changes. Persisting intermediate results makes analysis resumable and turns aggregate accuracy scores into concrete engineering work.
Jones also treats speed as an operational requirement: prompt optimization made one component of incident.io’s investigation agent four times faster without changing its intended behavior.
Looking beyond incident response, he anticipates company-specific coding agents working through permissioned internal MCP gateways, with identity controls, auditable tool access, and handoffs between specialized systems. His vision for engineering teams in 2027 places people in charge of judgment and deployment while investigation agents identify production failures and coding agents prepare potential fixes.
