Evaluations
- Average score
- 0.91+0.02vs last week
- Test cases
- 736+86added this week
- Regressions
- 22 suitesfell after a deploy
- Human review
- 12cases waiting for a person
Suites
| Suite | Agent | Score | Change | Last run | Result | Actions |
|---|---|---|---|---|---|---|
| Billing answers128 cases · rubric | Support copilot | 0.94 | +0.02 | 35 min agov14 | Passed | Open |
| Refund policy64 cases · rubric | Support copilot | 0.91 | +0.05 | 35 min agov14 | Passed | Open |
| Groundedness128 cases · rubric | Sales researcher | 0.84 | −0.07 | Oct 9v9 | Regressed | Open |
| Extraction accuracy240 cases · exact match | Invoice triage | 0.88 | −0.03 | Yesterdayv6 | Regressed | Open |
| Tone and safety96 cases · rubric | Onboarding guide | 0.97 | 0.00 | 2 days agov4 | Passed | Open |
| Review precision80 cases · human graded | Code reviewer | 0.90 | +0.01 | Nowv11 | Running | Open |
Regressions to review
2
Groundedness fell on Sales researcher0.91 to 0.84 after v9 · 13 cases got worseCompare versions
Extraction accuracy fell on Invoice triage0.91 to 0.88 after v6 · tax IDs are misreadOpen failing cases
Graders
- Rubric judge
- atlas-large
- Exact match
- 1 suite
- Human review
- 12 waiting