OctoAI

OctoAI built infrastructure for developers to run, customize and scale AI applications. Its hosted inference APIs supported language models, while OctoStack brought model serving, customization and asset management into customer-controlled cloud and on-premises environments. This let businesses deploy models alongside their own data. The company identified Apate.ai, Otherside AI, Latitude Games and Capitol AI as platform users; Apate.ai used OctoStack to run multilingual conversational models for combating telephone scams.

Founded in 2019 as OctoML, the company brought together Luis Ceze, Tianqi Chen, Thierry Moreau, Jared Roesch and Jason Knight. Co-founder and CEO Ceze now serves as NVIDIA’s VP of AI Systems Software. Its technical foundation was Apache TVM, a compiler project that predates the company. TVM combines graph and operator optimizations with learned cost models to search for efficient code across different hardware targets. OctoML commercialized that work to simplify model optimization and deployment across CPUs, GPUs and specialized accelerators.

OctoML closed a $15 million Series A in 2020, led by Amplify Partners with participation from seed investor Madrona. NVIDIA subsequently completed its acquisition of OctoAI. OctoAI’s commercial service wind-down took effect on October 31, 2024; its independent services are shut down, and Weaviate has deprecated and disabled the associated integrations.