Akshay Sharma is a machine learning engineer at Lyft building its self-serve AI agent platform for customer support. His work gives operations specialists more direct control over automated support while developing practical ways to determine whether conversational agents are ready for customers.
Earlier in his career, Sharma worked on TurboTax Desktop, developing an in-product e-commerce flow and an AWS-based system for analyzing application crashes. He studied at the University of Massachusetts Amherst between 2020 and 2022 and pursued projects in scientific-text analysis, document matching, sentiment modeling, self-supervised learning, and non-factoid question answering. His professional history spans application engineering, machine learning, and production conversational AI.
At Lyft, he helped develop a platform that lets domain experts configure support agents without relying on engineers for every policy or workflow change. His account of Lyft’s support architecture describes specialized agents, routing, tracing, monitoring, and evaluation built with LangGraph and LangSmith. The cross-functional initiative reduced agent-development timelines from approximately six months to a few weeks.
What makes agent evaluation useful
- Decision-relevant evaluation: Generic helpfulness or naturalness scores rarely tell teams what to fix. Sharma favors pass-or-fail criteria tied to specific outcomes, such as whether an agent explained a policy correctly or escalated an issue appropriately.
- Human-calibrated LLM judges: Automated graders should be validated like classifiers, using expert-labeled examples, separate development and validation datasets, and precision and recall measurements.
- Evaluation criteria drift: Effective rubrics evolve as production conversations reveal unfamiliar failures and expose disagreements between automated judgments and human expertise. Traces, annotation queues, and recurring error analysis keep evaluation grounded in actual behavior.
- Statistical confidence for launch decisions: Small improvements measured on limited samples can be noise. Representative sampling and confidence intervals become especially important when scores determine whether an agent ships.
In his AI Engineer World’s Fair session with Nick Ung, Sharma outlined evaluation systems that turn real customer interactions and expert feedback into concrete engineering decisions.
