Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curves

Key points

It pointed out that only a confidence band covering all budget points at once protects the choice of how many samples to draw, and computed the cost of existing fixed designs (192,000 answers for 100 questions and 64 budgets).
Since about three quarters of the variance in selected-answer accuracy at budget 64 comes from differences between questions, an audit that revisits every question need not pay that cost.
It built a paired audit based on an exponential inequality for two independent draws, reaching the minimum cost up to a logarithmic factor with no pilot.
In a real MMLU-Pro study it certified the whole curve with 79,133 answers, within 0.6% of what a cost law fitted beforehand predicted.

External research summaries. These are not HDATF publications or measured product results.