AI Quality Analyst

We run an AI-powered SMS bot and an AI-graded call training system for a private school network. Both systems have extensive test suites (400+ test files) and a RAG-based knowledge base that drives bot behavior.

We need someone to own the full quality cycle:

• Review production conversations to find bad bot responses
• Create test files (JSON) that reproduce the issue
• Fix the knowledge base content (Markdown files, WRONG/RIGHT examples)
• Run test suites to verify fixes and catch regressions
• Maintain and expand the knowledge base as our program evolves

This is NOT a software engineering role. The infrastructure is built. You're operating it — editing JSON, writing Markdown, running CLI scripts, reading test reports.

What You'll Work With

• JSON test files (you'll write and edit these daily)
• Markdown knowledge base documents