[plt-scheme] Native code generation and immutable pairs

On 2/10/06, Noel Welsh <noelwelsh at yahoo.com> wrote:
> Data is here:
>
>   http://citeseer.ist.psu.edu/serrano95control.html
>
> "Control Flow Analysis: a Functional Languages Compilation
> Paradigm", Serrano, M, 1995
>
> Using 0-cfa to infer types and hence remove type checks
> cuts execution time to about 60% of unoptimised code.  This
> is very interesting -- plenty of compiler papers are
> written about optimisations that get you only a 5-10%
> improvement.
>
> Correct statistical analysis of results has become my
> personal crusade in the last few days, so I feel compelled
> to point out there is no analysis of the significance of
> the results presented in the paper.  Which is to say we
> cannot say how 'lucky' the author would have to be to get
> these results by accident (I suspect the variance in
> execution time is very small, so they would probably hold,
> but it is annoying to see these simple steps not taken).

Well, since you've said that:

Reporting the percentage speedup from a given compiler optimization is
almost meaningless, because the number depends so much on the other
optimizations taking place along with it.  If you're starting with a
compiler that generates sufficiently lousy code, then it's not too
hard to find some optimization that will yield a 100% speedup (i.e.
double the rate at which the program executes, not make it execute in
zero time).  If you try out the same optimization in the context of a
compiler which already does similar stuff, then you may get a 0%
speedup.

Speaking from absolutely no real experience, I firmly believe that you
have to assess the value of an analysis or optimization in terms of
how well it will fit in and complement the other stuff you've already
got in place.