Sally-Ann DeLucia

Sally-Ann DeLucia is Director of Product at Arize AI and a core contributor to Alyx, the company’s AI engineering agent. She builds tools that help engineers investigate model behavior, plan complex work, and improve agents using evidence from their failures.

Her career began in data science. She earned a master’s degree in applied data science from Fairfield University and worked at New York Life Insurance Company and SuperAnnotate before joining Arize, where she previously worked as a machine-learning solutions engineer. As she moved into product leadership, she remained close to implementation, helping evolve Arize Copilot into assistance embedded in engineering workflows and contributing directly to Alyx.

Building agents that survive real use

• Context engineering as product design. Alyx analyzes observability traces containing prompts, tool calls, metadata, and other agents’ outputs; investigating those traces generates still more context. DeLucia found that aggressive truncation broke follow-up reasoning, while automated summaries discarded important details unpredictably. Her team adopted retrievable agent memory: preserve the beginning and end of messages, store omitted material, and let the agent recover it when necessary. She also tests long conversations by loading ten previous turns and evaluating the eleventh. Her account of managing agent context distinguishes this retrievable session history from durable cross-session memory, which remains a separate challenge.

• Subagent-based context isolation. Alyx delegates data-intensive searches and intermediate reasoning to specialized subagents, returning only the useful results to the main conversation. This keeps the user-facing interaction coherent while giving each analytical task its own working context.

• Planning beyond the chat interface. DeLucia’s writing on agent planning architecture and Alyx’s planning capabilities describes agents that interpret goals, sequence tasks, select tools, and connect debugging, evaluation, and prompt improvement into practical engineering workflows.

• Evaluation-driven prompt learning. In a workshop co-led with Fuad Ali, DeLucia explained how failed examples, evaluator explanations, and subject-matter feedback can guide prompt revisions more effectively than numerical scores alone. She also argues that useful domain specialization should not automatically be dismissed as overfitting.

• Co-evolving agent and evaluator loops. Because prompt optimization depends on trustworthy feedback, evaluators need their own review and improvement cycles. DeLucia extends testing beyond final answers to tool selection, planning, and intermediate decisions, and uses Alyx to improve Alyx through feedback drawn from the product’s actual behavior.