Agentic AI Application Tester (Night Shift) | 1x RTO/week

Responsibilities

• Definition and execution of testing and quality assurance strategies for AI-enabled workflows
• Continuous evaluation and monitoring of system behavior in production environments
• Contribution to auditability, risk management, and continuous quality improvement
• Define quality criteria and testing strategies for agent workflows, covering accuracy, latency, safety, compliance, and operational risk
• Build automated evaluation harnesses to assess agent performance, including hallucination rates, tool misuse, policy violations, and task success
• Implement continuous production monitoring to detect anomalies, quality degradation, and emerging safety concerns
• Develop and maintain automated test suites using Playwright for UI testing and custom scripts for API and workflow validation
• Apply LLM evaluation frameworks to assess output quality, regression, and system drift over time
• Pr...