Product Data Scientist (AI Evaluation & Quality)

Responsibilities
Si le interesa solicitar este empleo, por favor, asegúrese de cumplir los siguientes requisitos que se enumeran a continuación.

• 
  
• Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics
  
• Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency
  
• Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts
  
• Harden methodology: judge stability, non-determinism handling
  
• Translate numbers into decisions – weekly syncs, clear trade‑offs, no dashboards for their own sake
  

Qualifications

• 
  
• Solid foundation in statistics — sampling, hypothesis testing, variance, understanding what a noisy metric is
  
• Python and SQL — you can build an analysis end-t...