Mayan Soni, who works and publishes as Mayank Soni, is a senior AI engineer at Trainline building agentic travel assistance and improving how customer-facing AI systems are evaluated in production. His work bridges academic research into human conversation and the practical challenge of making automated travel support dependable.
Earlier, Soni researched neural dialogue systems at Trinity College Dublin’s ADAPT research centre. With Benjamin Cowan and Vincent Wade, he developed an inference-stage reranking approach that encourages conversational self-disclosure, helping dialogue models produce more engaging responses without retraining. He also coauthored research on topic transitions in Switchboard conversations, examining how discussion shifts and how unevenly participants initiate those changes.
In a study of ChatGPT-generated summaries, Soni and Wade combined blinded human assessment, automated metrics, and text classification. Human reviewers struggled to distinguish generated summaries from human-written ones even when classifiers detected differences, underscoring why fluent output alone cannot establish quality.
At Trainline, Soni applies that evaluation discipline to AI-assisted travel workflows. His work on production agentic systems centers on two operational priorities:
- Offline and online model evaluation: Compare replacement models before deployment, then monitor live performance to confirm that lower costs or improved efficiency do not degrade customer experience.
- Tool-call-level observability: Trace individual agent actions, model calls, and token usage to diagnose failures and assess complex workflows before and after release.
For Soni, conversational quality, operational visibility, and cost control are interconnected engineering problems: an AI assistant becomes trustworthy when its behavior can be inspected, measured, and improved under real production conditions.
