Engineering
How we cut a 9-minute suite to 41 seconds
Fan-out across 64 workers sounds easy until two cases share a rate limit. What we changed in the scheduler, and what we measured.
7 min
Product
Graders should check facts, not vibes
A judge model saying “looks right” is not a test. Why tool-calling graders now hit your staging API before they score a reply.
5 min
Guide
Your first eval suite in an afternoon
Pick ten real conversations, write the expected behaviour in plain YAML, and gate one pull request. A step-by-step walk-through.
9 min
Product
Why we price per run, not per seat
Seats punish the teams that test the most. The maths behind 10,000 free runs and $0.40 per 1,000 after that.
4 min
Engineering
Caching evals without lying to yourself
A cache hit is only safe if the prompt, the case and the model settings all match. How we hash all three, and the bug that taught us to.
8 min