Skipping unchanged cases is the biggest speed-up we have. It is also the easiest way to ship a regression, if the cache key is wrong.

We hash the prompt template, the case and every model setting. Change the temperature and every case reruns.