The Automation QA Engineer defines, implements, and continuously improves quality assurance practices for AI enabled and agent based solutions. The role ensures systems meet agreed standards for accuracy, reliability, performance, safety, and compliance, and supports operational readiness through automated evaluation and monitoring.
The role:
Definition and execution of testing and quality assurance strategies for AI enabled workflowsContinuous evaluation and monitoring of system behavior in production environmentsContribution to auditability, risk management, and continuous quality improvementDefine quality criteria and testing strategies for agent workflows, covering accuracy, latency, safety, compliance, and operational riskBuild automated evaluation harnesses to assess agent performance, including hallucination rates, tool misuse, policy violations, and task successImplement continuous production monitoring to detec...