Rafal Wilinski is a founding engineer at Runlayer, the founder of Dynobase, and a former technical lead for Zapier Agents. He builds tools that make complex infrastructure and AI agents easier to operate, evaluate, and trust.
Wilinski began by developing mobile games before moving into web applications, cloud infrastructure, and open source. He created express-status-monitor, a real-time monitoring tool for Node.js and Express, and contributed Java and Kotlin support to the Serverless Framework.
As a cloud engineer at X-Team from 2017 to 2020, he encountered the frustrations of inspecting DynamoDB records, navigating unfamiliar database structures, and switching between AWS accounts and regions. In 2019, he founded Dynobase, a graphical DynamoDB client that simplifies querying, editing, and working with single-table designs. He subsequently joined Stedi as a serverless engineer, building fault-tolerant, multiregion AWS systems from 2020 to 2022.
Wilinski joined Vendr in 2022, helping establish its AI team and leading generative-AI projects. At Zapier, where he became technical lead for Zapier Agents in 2024, he worked on software that translates users’ requests into automated business processes. The challenge was making probabilistic models dependable when they interact with external tools, consequential side effects, and unpredictable users.
He joined Runlayer as a founding engineer in September 2025, extending that work into enterprise agent infrastructure, including permissions, observability, security, and operational control. His work on Baseten-hosted models for Runlayer agents describes model selection per agent alongside identity-aware governance and cost controls.
- The agentic evaluations pyramid: Wilinski’s three-layer evaluation framework combines focused checks for individual failures, trajectory evaluations of complete multistep workflows, and live A/B testing. Narrow tests can punish stronger models for reaching the same outcome differently; trajectory-based grading considers tool calls, generated artifacts, and final results. Realistic synthetic environments allow teams to test consequential workflows without sending actual customer emails or altering production systems.
- Production failures as evaluation data: His agent-testing approach combines generated scenarios with execution traces, customer feedback, observed failures, and security tests. At the 2025 AI Engineer World’s Fair, he and Vitor Balocco described how Zapier incorporated explicit feedback, behavioral signals, frustration analysis, and business metrics into agent evaluation. A growing test suite may lower benchmark scores while improving the product by exposing harder real-world failures.
- Practical retrieval-augmented generation: His retrieval guidance emphasizes document-aware chunking, metadata, hybrid search, human review, and separating retrieval representations from answer-generation context. His cloudflare-rag project applies full-text and vector search, query rewriting, reciprocal-rank fusion, and streamed responses.
- Model Context Protocol integrations: His aws-mcp connects assistants to AWS resources through locally configured credentials, while mcp-apple-notes enables semantic search across Apple Notes using local execution and on-device embeddings.
Wilinski’s principles for reliable agents begin with a basic architectural constraint: avoid autonomous agents when a simpler model call will suffice. When autonomy is warranted, separate planning from execution, enforce security boundaries, preserve human oversight, recover from tool failures, and make behavior observable.
