Found Description
Applied AI Engineer — Evaluation & Reliability The problem you'd own 15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line — by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.
We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't prompt harder. It's to route every number through deterministic computation and gate every report behind checks that pass or fail — never a model grading its own work. You'd build that layer.
What you'll build
Deterministic evaluation gates that block a report when a defect is present — measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies ...
We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't prompt harder. It's to route every number through deterministic computation and gate every report behind checks that pass or fail — never a model grading its own work. You'd build that layer.
What you'll build
Deterministic evaluation gates that block a report when a defect is present — measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies ...
Ready to Apply?
Submit your application for Applied AI Engineer — Evaluation & Reliability at 15Rock
Apply Now