# [plt-scheme] Statistics (V301.5 Speed Up)

--- Jim Blandy <jimb at red-bean.com> wrote:
>* As far as beer is concerned: my interpretation of Joe's
*>* point was that
*>* we should see a difference in the tails of the
*>* distributions: while
*>* the right tail (runs taking more time) should go on
*>* arbitrarily (maybe
*>* the machine decides to go off and do a Sudoku while it's
*>* running your
*>* benchmark), there should be a point on the left below
*>* which we see no
*>* outliers. What do folks think?
*
Yes, and that's pretty much a Poisson distribution.
Imagine you have 1000 computers. You set them all running
a benchmark. Every 10ms (your timer resolution) you check
to see if any benchmarks have finished. When all have
finished you can plot the results, and I'd expect the them
to follow a Poisson distribution (as the definition of a
Poisson distribution is more or less the model I've
described above). Now note that a Normal distribution is
an excellent approximation to a Poisson when the mean time
to completion is large. See, e.g., Wikipedia:
http://en.wikipedia.org/wiki/Poisson_distribution
So for little micro benchmarks I'd expect a
Poisson/binomial distribution. For larger benchmarks a
Normal distribution is a good enough approximation. How
can we tell if the models match the reality? There are a
number of statistical tests for fit one can do. E.g. the
Kolmogorov-Smirnov test.
For the bimodal results Doug posted, it looks very much
like a mixture of Gaussians to me. That's interesting in
itself -- what is happening to cause this behaviour? The
GC has been hypothesised. It should be possible to verify
this.
This is, at least to me, interesting stuff. If I get time
I'll write more, possibly a workshop publication.
N.
Email: noelwelsh <at> yahoo <dot> com noel <at> untyped <dot> com
AIM: noelhwelsh
Blogs: http://monospaced.blogspot.com/ http://www.untyped.com/untyping/
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com