Eugene Yan

Eugene Yan is a member of technical staff at Anthropic who builds safe, reliable AI systems. His career spans e-commerce recommendations, Amazon search and generative AI, and cybersecurity, with a consistent emphasis on evaluation-driven development: measure whether systems work under real conditions before trusting their outputs.

Yan studied psychology and business at Singapore Management University and began his career working on policy and trade agreements at Singapore’s Ministry of Trade and Industry. After developing his technical skills, he joined IBM, working on workforce analytics, fraud-related problems, job-demand forecasting, and recommendations.

A product-classification project helped bring him to Lazada, where he became vice president of machine learning and built systems for ranking, push notifications, and automated classification. He subsequently led machine learning at healthcare startup uCare.ai, including work on disease detection and healthcare-cost estimation. At Amazon, he rose to principal applied scientist, developing real-time retrieval, bandit-based ranking, search recommendations, and AI products for summarization, translation, and question answering. He joined Anthropic in 2026.

What makes AI dependable

• Evaluation-driven development: Yan favors realistic, task-specific tests over broad benchmark scores: execute generated SQL, validate extracted JSON, and check whether individual sentences in a summary follow from their source. His product-evaluation framework starts with representative labeled examples, calibrates automated judgments against human preferences, and reruns evaluations after system changes. His analysis of model-based evaluators examines ordering effects, bias, and the tradeoffs between flexible language-model judges and faster specialized classifiers.

• Retrieval and feedback must reflect actual behavior: Relevant documents can become ineffective when buried among distractions, and larger context windows do not rescue poor ranking. Yan applies recommendation-system principles to retrieval: prioritize relevant evidence, exclude weak matches, and distinguish genuine user satisfaction from misleading signals such as copying incorrect generated code. His production-systems talk connects retrieval quality, factuality guardrails, and interfaces that naturally capture useful feedback.

• Semantic identifiers improve discovery: Conventional item identifiers reveal little about new or rarely encountered products. Yan’s semantic-ids-llm project explores semantic identifiers that connect item meaning with language-model reasoning, enabling more steerable recommendations. His recommendation-systems keynote also examines synthetic labels, multimodal representations, and unified search-and-recommendation models, carefully distinguishing external company examples from his own work.

• Security agents need independent verification: Yan’s source-code security workflow comprises threat modeling, isolated sandboxes, vulnerability discovery, exploit verification, triage, and patch validation. Discovery optimizes for recall; an independent verifier tests suspected vulnerabilities in a fresh environment to establish precision. Patches must stop the original exploit without breaking existing tests, while severity assessments incorporate operational context such as firewalls, internal access, and sensitive data. As Yan observed publicly, stronger models shift the security bottleneck from finding vulnerabilities toward verifying, prioritizing, and fixing them.

Yan also makes production engineering knowledge reusable. His Applied Machine Learning repository collects papers and technical accounts of deployed systems, while Machine Learning Design Documents shares practical planning examples. His more recent writing on working with AI extends those concerns to human-model collaboration: durable context, feedback, and reliable performance on difficult tasks determine whether capable models translate into finished work.