Vitor Balocco is co-founder and chief architect of Runlayer, where he builds infrastructure that helps enterprises connect AI agents to business applications while controlling access, sensitive data, and consequential actions. Before founding the company with Andrew Berman and Tal Peretz, he was a staff AI engineer at Zapier and technical lead for Zapier Agents.
Balocco studied at the Pontifical Catholic University of Rio de Janeiro and became a maintainer of ESLint, the widely used JavaScript linting project. His contributions included configurable diagnostic messages and corrections for inaccurate linting results. At Zapier, he helped create the AI Zap builder, which turns natural-language requests into automated workflows, and contributed to Zapier Agents, Zapier MCP, and AI custom actions.
His approach to assertion-based agent evaluation, documented in a technical guide to evaluating tool-using assistants, tests observable behavior without demanding identical wording. An assistant should request missing information, choose appropriate tools, pass sensible arguments, and acknowledge unavailable capabilities.
By the 2025 AI Engineer World’s Fair, Balocco and fellow Zapier engineer Rafal Wilinski were applying those ideas to unpredictable production environments. Their work on real-world agent failures combined execution traces, tool-call errors, explicit feedback, behavioral signals, and recurring customer frustrations to create evaluation datasets tied to actual product outcomes.
- Trajectory-aware evaluation: Judge an agent’s complete sequence of tool calls, intermediate artifacts, and final results. Narrow checks can wrongly penalize stronger models that solve the same problem differently; human-written rubrics and controlled experiments help connect evaluation scores to user satisfaction.
- Prompt injection and privileged access: Balocco’s analysis of a GitHub MCP vulnerability shows how malicious public-repository content can manipulate an agent with access to private information. His proposed defenses include least-privilege permissions, session-specific restrictions, credential scanning, approved tools, human confirmation, and monitored gateways.
- Governed interfaces for production agents: His comparison of MCP and command-line tools favors constrained execution surfaces that make agent operations auditable, restrict permissions by session, and validate potentially consequential actions.
