Found Description
What You Will Do
- Define, track, and report on AI performance metrics: resolution rates, deflection rates, escalation accuracy, CSAT, and response quality
- Build and maintain AI performance dashboards and reporting infrastructure
- Conduct regular evaluations of AI agent outputs to identify accuracy failures, hallucinations, and behavior regressions
- Design and run controlled experiments (A/B tests) to evaluate the impact of AI changes on performance metrics
- Partner with AI Engineers and the AI Support Architect to diagnose performance issues and recommend interventions
- Work with Intent & Demand Analysts to connect intent data with AI resolution outcomes
- Produce regular performance reviews and present insights to Knowledge & AI leadership
- Develop evaluation frameworks and test sets for ongoing AI quality assurance
- Monitor AI performance trends over time and flag regressions or deterioration early