The first version of parallel runs split a suite into equal chunks. It was fast on paper and slow in practice, because a handful of slow cases always landed together.

We now schedule by measured case duration, and every worker gets its own token bucket for the model provider. The result on our largest customer suite: 9 minutes 12 seconds down to 41 seconds.