Principal Data Scientist - Agent Builder

Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale — unleashing the potential of businesses and people.

What Is The Role

As a Principal Data Scientist on the Search Conversational Experiences team, you will help set the technical direction for evaluating, improving, and scaling chat quality across Elastic’s agentic platform. You will define the evaluation strategy that guides product decisions, including models, routing, tools, and tailoring agents to use cases. You will work closely with backend engineering, product, UX, and other data scientists to turn ambiguous, cutting‑edge problems into measurable product improvements.

What You Will Be Doing

• Define the evaluation strategy for conversational and agentic search, including offline and online evaluation, golden datasets, rubrics, LLM‑as‑judge calibration, groundedness and citation checks, and A/B testing.
• Lead ...