Deanna Emery

Deanna Emery is an AI researcher at Databricks and the former founding AI researcher at Quotient AI, specializing in the reliability of language models, search systems, and AI agents. She led the development of HalluMix, an open benchmark that tests whether hallucination detectors recognize unsupported claims amid the ambiguous, noisy information encountered by real applications.

Emery began her research career in astrophysics at Harvard and the Smithsonian Astrophysical Observatory, leading research into a galaxy-cluster collision surrounding the radio galaxy 3C 438. At Aon, she became a principal data scientist, leading work on language models for intellectual-property valuation and contributing to a patent application for dataset-distinctiveness modeling. She completed the University of California, Berkeley’s Master of Information and Data Science program in 2023, collaborating on SignSense, an American Sign Language translation project.

• HalluMix and hallucination detection. Emery was the lead author of the HalluMix research paper, which introduced a 6,500-example benchmark spanning healthcare, law, science, and news. Its public dataset tests detectors across question answering, summarization, inference, distracting documents, and varying context lengths, exposing meaningful tradeoffs between precision and recall.

• Evaluation-driven retrieval optimization. Emery demonstrated how targeted experiments can improve retrieval-augmented generation by adjusting document chunking, retrieval windows, embedding models, reranking, and hybrid search over sparse and dense vectors. Her results distinguished retrieving necessary evidence from introducing irrelevant context: larger chunks improved evidence coverage while reducing faithfulness, whereas smaller chunks, broader retrieval, and reranking produced more grounded answers.

• Reference-free evaluation for live search. For continuously changing web information, Emery evaluates answer completeness, document relevance, and unsupported claims without predefined ground-truth answers. These measures expose distinct failures: a thorough answer can hallucinate, relevant sources cannot guarantee faithful reasoning, and citations without underlying documents obstruct diagnosis.

In 2026, Emery joined Databricks through its acquisition of Quotient AI, extending her work on continuous production evaluation and more dependable enterprise AI agents.