Philip Kiely is Head of AI Education at Baseten, an early employee of the model-inference company, and the author of Inference Engineering. He helps engineers understand how model architecture, GPU hardware, serving software, networking, and application design determine the speed and cost of production AI.
Kiely began publishing technical tutorials in 2019 while studying computer science at Grinnell College. He wrote for Smashing Magazine, Twilio, and other technology publications, published Writing for Software Developers shortly before graduating with an honors degree in 2020, and created Who Pays Technical Writers to connect writers with paid opportunities.
He joined Baseten in January 2022 as its tenth employee, working across developer relations, technical content, documentation, and education. As generative AI moved into production, he concentrated on inference: choosing hardware, deploying models, measuring bottlenecks, and balancing latency, throughput, reliability, and cost. His 2026 book Inference Engineering maps that discipline across model architecture, GPUs, serving runtimes, optimization, multimodal applications, and production infrastructure.
- Inference performance depends on the workload. GPU type, batch size, sequence length, concurrency, and latency targets determine which configuration actually performs well. Kiely emphasizes representative benchmarks, tail latency, and the difference between throughput experienced by one user and total GPU output. His TensorRT-LLM workshop also addresses hardware-specific engines, quantization, deployment, and autoscaling constraints.
- Open serving frameworks expand practical choices. Kiely works across TensorRT-LLM, SGLang, and vLLM, choosing among hardware optimization, extensibility, and model compatibility. His SGLang workshop connects runtime selection to speculative decoding, realistic benchmarking, inspectable implementations, and open-source contributions.
- Real-time voice inference changes the optimization target. For Orpheus text-to-speech, roughly 83 tokens per second satisfies real-time playback; first-byte latency, simultaneous streams, and GPU utilization then matter more than additional token throughput. Kiely combines FP8 quantization, TensorRT-LLM, GPU-based audio decoding, and dynamic batching while accounting for connection reuse, network placement, and coordination across listening, reasoning, and speech systems.
- Technical books can become reusable audio pipelines. Kiely built an AI-narrated audiobook of Inference Engineering using Rime’s Coda model and a custom clone of his voice. The project involved rewriting diagrams and tables for audio, segmenting the manuscript, correcting pronunciation, generating speech, and assembling audiobook files.
