← All speakers

Bio, Work & Ideas

Ankur Goyal

Conference affiliation: Braintrust · 2025

Ankur Goyal is the founder and chief executive of Braintrust, which builds evaluation and observability infrastructure for AI applications. He treats AI evaluation as production infrastructure: a continuous feedback loop connecting model behavior, user experience, product requirements, and changing technical capabilities.

Goyal studied computer science at Carnegie Mellon and worked on distributed databases at MemSQL, later renamed SingleStore, where he became vice president of engineering. He subsequently founded Impira, an AI company focused on extracting information from documents. After Figma acquired Impira in 2023, he led its AI team. Building evaluation systems at both companies inspired Braintrust, whose earliest users included Zapier. In February 2026, the company announced an $80 million Series B.

  • Evaluation should determine what gets built. Goyal judges evaluations by whether they help teams investigate customer complaints, assess potential features before release, and adopt improved models quickly. He advocates testing ambitious capabilities that current models cannot handle, allowing developers to recognize immediately when a new release makes them practical. His practical framework for evaluations treats those tests as instruments of product discovery, not merely regression checks.
  • Datasets and scoring functions encode product judgment. Evaluation datasets must evolve with real customer behavior, while scoring functions should reflect an application’s specific requirements. Braintrust’s open-source Autoevals library supplies reusable evaluators, but Goyal expects advanced teams to customize their criteria. He also favors human review before converting complaints into test cases: dissatisfaction does not necessarily mean an answer was incorrect.
  • Context engineering extends to agent tools. An agent’s effectiveness depends on its complete context, including tool definitions, tool responses, and accumulated interactions. Goyal describes an internal experiment where replacing JSON with YAML improved a model’s ability to use a tool’s output, illustrating why interfaces designed for software may require redesign for language models.
  • Loop optimizes the whole evaluation system. Braintrust’s Loop evaluation agent improves prompts, datasets, and scoring functions together, presenting proposed edits visibly and supporting multiple model providers. Its premise is that weak results can originate in incomplete examples or flawed grading as readily as in the prompt.
  • Active observability makes production traces usable. Goyal’s more recent work on active observability emphasizes automatically clustering conversations, identifying recurring patterns, and surfacing problems in production. Braintrust’s Topics capability addresses the associated challenges of reconstructing conversation threads, managing inference costs, and analyzing large trace volumes.

Read the topics behind these talks

3 conference talks

AI Engineer World's Fair 202414:59

How Zapier Builds AI Products and Features With the Help of Braintrust

Zapier’s Olmo Maldonado and Braintrust’s Ankur Goyal explain how AI Zap Builder and Zapier Copilot were developed around systematic evaluations and observability. They describe synthetic evaluation data, CI and manual test runs, provider load testing, logic-based and LLM-based graders, and rapid regression detection. The presentation also contrasts GPT-4…

Ankur Goyal · Olmo Maldonado

Coding and developer tools · Observability and reliability · Evals

References