[racket] Computer Language Benchmark Game

Yesterday, Isaac Gouy wrote:
> Eli Barzilay <eli at ...> writes:
> -snip-
> > > seems to be measuring gmp performance rather than racket's...
> > 
> > Yes, it's unfortunate, but that's what practically everyone does.
> 
> There's even a Racket program that doesn't use GMP!
> 
> http://shootout.alioth.debian.org/u32/program.php?test=pidigits&lang=racket&id=1

Yes, there is, and it is slower for obvious reasons.


> > BTW₂, the whole shootout thing is generally biased and almost
> > completely useless for drawing any kind of conclusions.  Really.
> > (I mean it in a stronger sense than usual benchmark disclaimers.)
> 
> "generally biased"? 

When submissions are dropped because of a vague "it's too fast",
that's a bias.  It's a pity since the shootout started as a showcase
of functional languages, and being able to factor out parts of code to
optimize it is the main advantage of such languages.  Arguably the
*only* one, since given enough effort it's always possible to do any
of these optimizations in C.  The showcase bit is in the ability to
recognize and quickly implement refactoring and optimization
opportunities -- to the point that a C programmer wouldn't even think
about doing these things.

Memoization is a perfect example of this phenomena, which goes well
beyond C -- "theory people" go straight to conventional dynamic
programming, which gets hairy enough that it's considered a heavy
tool.  As a result, it is not used nearly as much as it could,
something that gets even worse when these people use a conventional
language like C, or write C in some other language.  The perfectness
of the memoization example is something that has lingered on for
*decades*, I've seen it happen time after time.  I've seen people who
were so blind to this that they reached ridiculously bogus
concnclusions about memoization-based algorithms being NP-SPACE.
Obviously, it is possible to do memoization in C too -- but it's just
not easy enough to quickly try it out, and that's exactly where good
functional languages shine.  After all of these decades, things like
memoization, and these advantages of such languages are finally
becoming a bit more known, but it took a collectively huge effort to
get there.

Now take a bunch of problems and throw them at a crown that tries to
compete for speed.  In such a limited ecosystem the feedback loop is
much shorter and the propagation of fast solutions is much more
effective.  That makes such competitions mostly nonsensical, since
that conceptual advantage of functional languages is practically lost.
That's not bias, it's the nature of things.  But when such solutions
are *disqualified* and specs change to *forbid* them, then functional
languages lose this single advantage and get into a perpetual game of
mimicking C solutions.  To get noticed at *that* game you need people
who know how to penetrate through all of these well-crafted layers of
useful abstractions in their well-designed language.  Notice how
things flip here: instead of FP programmers quickly coming up with new
ways to solve problems efficiently and C programmers trying to catch
up by implementing significant parts of these languages -- instead of
this, we're thrown into an inverse world where it's the FP programmers
that labor over C-like solutions, with the required extra layers of
verbosity.  The competition factor is becoming useless with this, and
more than that, the potential aspect of finding a good solution for
practical problems is lost when these problems get specified down from
"solve this problem fast" to the point of "execute this exact
computation N times".  All of this is bias.  (And it's the bad kind of
bias, one where one side is completely unaware of it.  All they know
is that "memoization" is some kind of black magic that is obviously
cheating, and "obviously" we need to make sure that such cheating
doesn't happen and demand that no such tricks are played.)


> Is there some way you think that differs from kindergarten
> name-calling?

Yes.  Please take petty flaming attempts elsewhere.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!