Emma Ning is a principal program manager at Microsoft Foundry who develops tools for running AI models on personal computers, browsers, and other edge devices. Her contributions to ONNX Runtime Web, Olive, and Foundry Local address a central challenge of on-device AI: delivering responsive, private applications across different hardware without requiring developers to master every processor and optimization technique.
By 2020, Ning worked in Microsoft’s Azure Machine Learning organization on optimizing transformer inference across CPUs and GPUs. As a senior product manager, she also advanced browser-based inference with ONNX.js, using WebAssembly and WebGL to support privacy-sensitive, offline-capable applications.
In 2021, she helped introduce ONNX Runtime Web and worked on deploying transformer models for code completion in Visual Studio and Visual Studio Code. She subsequently co-introduced Olive’s hardware-aware optimization toolchain in 2023 and helped bring WebGPU-accelerated generative AI to browsers in 2024.
Her priorities include:
- Local inference as a product requirement: Offline reliability, confidential documents, real-time responsiveness, and inference costs can make on-device execution essential.
- Hardware-aware model optimization: Quantization, runtime tuning, and processor-specific acceleration must balance speed against accuracy while remaining accessible to application developers.
- Browser-native generative AI: WebAssembly, WebGL, and WebGPU allow familiar JavaScript applications to execute increasingly sophisticated models on users’ devices.
- Permission-aware local agents: Small models can use explicitly authorized filesystem and optical-character-recognition tools, provided their tasks reflect the limitations of on-device inference.
With Foundry Local, Ning brought these priorities together in an application stack combining ONNX Runtime, downloadable models, hardware acceleration, command-line tools, and developer SDKs for Windows and macOS. Her demonstrations compared smaller models’ speed and output quality, summarized sensitive documents locally, and configured agents to inspect files with user permission. More recently, her work has extended Foundry Local into edge and enterprise environments through speech transcription, embeddings, tool calling, and broader Linux, ARM, and on-premises support.
