Rajat Shah is a staff software engineer at Netflix working on its Model Inference Platform, infrastructure supporting personalized recommendations, search, fraud detection, and other machine-learning applications. He designs systems that route enormous volumes of inference traffic and develops AI-assisted techniques for finding costly production bottlenecks.
Shah studied computer science at Visvesvaraya National Institute of Technology in India, interned at Goldman Sachs, and worked at Morgan Stanley before earning a computer-science master’s degree at North Carolina State University. His graduate research included automated short-answer grading, incorporating information about students, questions, and responses into machine-learning classifiers.
At Amazon from 2017 to 2021, Shah worked on imaging technology, search, and recommendations, including deep-learning-based language identification for search queries. He also created Amazon Gather, an internal community and event-management platform that reached more than 500,000 users by the end of 2020. That October, he received Amazon’s Just Do It award from Jeff Bezos for building Gather.
After joining Netflix in 2021, Shah helped develop centralized model-serving infrastructure that, by 2025, handled approximately one million requests per second. His engineering interests encompass several related challenges:
- Inference routing at scale. Shah co-authored Netflix’s account of its model-serving architecture, describing how applications request business outcomes without managing specific model versions or compute clusters. The original Switchboard routing service simplified experimentation but introduced a shared failure point and an extra network hop; its successor, Lightbulb, separates model-selection metadata from traffic forwarding through Envoy.
- AI-assisted performance engineering. Shah uses CPU profiles and call stacks to help agents identify expensive code, locate the exact Git revision running in production, and propose targeted fixes. One investigation uncovered a quadratic-time operation consuming 8.8% of sampled CPU; another found repeated metrics allocations across seven services. His performance-engineering workflow prioritizes measurable production improvements over plausible-looking suggestions.
- Shared performance-pattern catalogs. Shah advocates recording optimization patterns and anti-patterns as Markdown in Git, giving agents reusable organizational knowledge across services and languages. That catalog can support production investigations, flag inefficient changes during review, and guide coding agents before problematic implementations ship.
- Canary validation and human approval. Automated tests check functional correctness, while canary deployments compare CPU consumption, latency, and error rates under real traffic. Engineers retain final approval; greater agent autonomy requires stronger evaluation, sandboxing, and protection against prompt injection.
