Ameya Bhatawdekar

Ameya Bhatawdekar is vice president and field chief technology officer at Braintrust, helping organizations make AI applications reliable through production-grade AI evaluation and observability. Previously Dropbox’s director of machine learning, he helped build Dropbox Dash and developed practical methods for measuring conversational systems whose failures can originate anywhere from document retrieval to model reasoning.

From online safety to production AI

Bhatawdekar worked on defenses against socially engineered attacks at Microsoft and abuse prevention at Facebook AI, coauthoring research on gaps between harmful-content detection research and platforms’ moderation needs. He later applied machine learning to Microsoft Office, contributing to Semantic Interpreter, which translates natural-language requests into executable actions inside productivity applications.

At Dropbox, he helped develop Dash’s AI-powered search and conversational features. In a 2025 blueprint for evaluating conversational AI at scale, he and his coauthors described replacing informal testing with curated datasets, calibrated model-based judges, automated release gates, production monitoring, and user feedback. Braintrust was already part of Dropbox’s evaluation infrastructure before Bhatawdekar joined the company as VP and field CTO.

How he thinks about reliable agents

• Evaluation must follow architecture. Single-prompt systems can be judged largely by their answers; retrieval systems require checks on context, workflow graphs require routing and node-contract tests, and autonomous agents introduce new risks through memory, tools, code execution, and external actions. An incident-response agent that can roll back deployments demands stronger safeguards than one that only answers questions.

• Capability is different from reliability. An agent’s first success demonstrates what it can accomplish; repeated runs reveal whether it performs consistently across different execution paths. Bhatawdekar advocates distribution-based agent evaluation to measure both dimensions, especially when agents can modify external systems.

• Production failures should become future tests. His production-to-evaluation flywheel converts real-world traces and newly discovered failure patterns into regression cases and expanded evaluation datasets. Braintrust’s Topics feature clusters production behavior to surface problems existing tests overlooked, keeping coverage aligned with changing agent capabilities.

Bhatawdekar also chairs the scientific advisory board of National PKU News and serves on its governing board.