[plt-scheme] Perplexed Programmers

I hate to spoil a good, old-fashioned flame-fest on the terrible state
of computing practice, but I have a hard time agreeing with much of
what I read here.

To those of you firmly in the negative ("software crisis") camp, there
are two things you might mean:

1. Software is in a terrible state, period.

2. Software is in a parlous state relative to the practice of other,
more mature, engineering disciplines.

It's easy to find all sorts of evidence for #1 (I know, I trot it all
out to scare my students)...except I just don't find it very
convincing.  I just think back to my own life a mere ten years ago,
compare to where it is now, and I am periodically *amazed* at what
software can do for me, and how more it promises to do.  Of course
it's not all just "software", but parallel advances in hardware,
databases and storage, algorithms, statistical methods, and so on --
but "software" has always necessarily been a surrogate for all those
things.  Tell me: is the software world you inhabit not better than
what it was two decades ago?  (Yes, even you Lisp Machine people are
welcome to try disagreeing...try.)

As for #2, I think we have too hallowed an estimate of traditional
engineering practice.  I confess I haven't chased down the numbers on
completion rates, on-time-under-cost rates, and so on for projects in
bricks-and-mortar engineering.  But at the edges, nobody is doing too
well, from space shuttles -- who can forget Gregg Easterbrook's
absolutely phenomenal 1980 Washington Monthly essay on them, before
they had even gone into space -- to bridges (now they're thinking it
may be *bird poop*?!?).

I'll tell you what I think the "software crisis" is.  It's that the
difficult stuff is possible but virtually none of the easy stuff is
easy.  There are orders-of-magnitude differences in the effort and
memory required to do simple tasks in different tools, and even the
easiest of them sometimes require my time and mental space to too many
significant digits.  Use an "baby" tool and you're locked in to a
place where you can't do anything hard; use a "pro" tool and you are
so weighed down you can't get off the damn ground, never mind
achieving escape velocity.

I think there are two things we're failing at.  One is to shift
between intensional and extensional representations -- sometimes
working top-down with programs, sometimes working bottom-up with
examples (though don't bother pointing me at "programming-by-example"
systems, which I think largely miss the bidirectionality) -- and the
other is to use all those cycles in the environment to actually do
some of my thinking: not just create name-completion lists but to do
something actually *hard*, whether ask me questions or tell me about
things I didn't realize or suggest problems I didn't anticipate.

The real tragedy of the past 20 years is not that the Lisp Machine
didn't grow by two orders of magnitude (cue obligatory joke about
global warming) but that the Programmer's Apprentice didn't.

Shriram