[racket] Computer Language Benchmark Game

From: Eli Barzilay (eli at barzilay.org)
Date: Sat Feb 18 22:29:46 EST 2012

Sorry for the seemingly out-of-nowhere aggressive reply below.

To explain some of the background, Issac is the person who is running
the shootout at the moment.  We have had private email conversations
on this topics before, and I let my frustration from that context show
through in a bad way.

In any case, I didn't mean to make this Racket seem like a less
welcoming place.

Yesterday, Eli Barzilay wrote:
> Yesterday, Isaac Gouy wrote:
> > Eli Barzilay <eli at ...> writes:
> > -snip-
> > > > seems to be measuring gmp performance rather than racket's...
> > > 
> > > Yes, it's unfortunate, but that's what practically everyone does.
> > 
> > There's even a Racket program that doesn't use GMP!
> > 
> > http://shootout.alioth.debian.org/u32/program.php?test=pidigits&lang=racket&id=1
> Yes, there is, and it is slower for obvious reasons.
> > > BTW₂, the whole shootout thing is generally biased and almost
> > > completely useless for drawing any kind of conclusions.  Really.
> > > (I mean it in a stronger sense than usual benchmark disclaimers.)
> > 
> > "generally biased"? 
> When submissions are dropped because of a vague "it's too fast",
> that's a bias.  It's a pity since the shootout started as a showcase
> of functional languages, and being able to factor out parts of code to
> optimize it is the main advantage of such languages.  Arguably the
> *only* one, since given enough effort it's always possible to do any
> of these optimizations in C.  The showcase bit is in the ability to
> recognize and quickly implement refactoring and optimization
> opportunities -- to the point that a C programmer wouldn't even think
> about doing these things.
> Memoization is a perfect example of this phenomena, which goes well
> beyond C -- "theory people" go straight to conventional dynamic
> programming, which gets hairy enough that it's considered a heavy
> tool.  As a result, it is not used nearly as much as it could,
> something that gets even worse when these people use a conventional
> language like C, or write C in some other language.  The perfectness
> of the memoization example is something that has lingered on for
> *decades*, I've seen it happen time after time.  I've seen people who
> were so blind to this that they reached ridiculously bogus
> concnclusions about memoization-based algorithms being NP-SPACE.
> Obviously, it is possible to do memoization in C too -- but it's just
> not easy enough to quickly try it out, and that's exactly where good
> functional languages shine.  After all of these decades, things like
> memoization, and these advantages of such languages are finally
> becoming a bit more known, but it took a collectively huge effort to
> get there.
> Now take a bunch of problems and throw them at a crown that tries to
> compete for speed.  In such a limited ecosystem the feedback loop is
> much shorter and the propagation of fast solutions is much more
> effective.  That makes such competitions mostly nonsensical, since
> that conceptual advantage of functional languages is practically lost.
> That's not bias, it's the nature of things.  But when such solutions
> are *disqualified* and specs change to *forbid* them, then functional
> languages lose this single advantage and get into a perpetual game of
> mimicking C solutions.  To get noticed at *that* game you need people
> who know how to penetrate through all of these well-crafted layers of
> useful abstractions in their well-designed language.  Notice how
> things flip here: instead of FP programmers quickly coming up with new
> ways to solve problems efficiently and C programmers trying to catch
> up by implementing significant parts of these languages -- instead of
> this, we're thrown into an inverse world where it's the FP programmers
> that labor over C-like solutions, with the required extra layers of
> verbosity.  The competition factor is becoming useless with this, and
> more than that, the potential aspect of finding a good solution for
> practical problems is lost when these problems get specified down from
> "solve this problem fast" to the point of "execute this exact
> computation N times".  All of this is bias.  (And it's the bad kind of
> bias, one where one side is completely unaware of it.  All they know
> is that "memoization" is some kind of black magic that is obviously
> cheating, and "obviously" we need to make sure that such cheating
> doesn't happen and demand that no such tricks are played.)
> > Is there some way you think that differs from kindergarten
> > name-calling?
> Yes.  Please take petty flaming attempts elsewhere.

          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Posted on the users mailing list.