[plt-scheme] [maybe off-topic] The theory of testing

I think in my entire life there was only a brief period when I was  
young when I managed to crank out code in high volume without lots of  
tests and miraculously everything worked.  The rest of the time I've  
been deservedly bitten for such recklessness.  We are all human beings  
and the reality is we need something to check the soundness of our  
coding with wild abandon.  Every time I write code quickly without  
bothering to test my assumptions I just know I'm asking for trouble  
and when I end up in the debugger I usually find something really  
stupid that would easily have been averted by slowing down a little,  
thinking more, and writing a contract and a test.

As a pilot I wouldn't think of taking off without pre-flight checks-- 
you're never supposed to ASSume anything, you're supposed to use a  
checklist (design recipe).  Lives depend on it.  The FAA, insurance  
companies and lawyers eat you alive if you don't check before you leap.

If a project fails, it doesn't prove unit tests are pointless;  
probably the project wasn't what the customer wanted or there was a  
higher level design flaw.

Unit tests don't guarantee perfection, they just improve the odds.  In  
my code, if I write more than a trivial amount of code and don't write  
tests, I'd say there's an 80% chance that something is broken.  If I  
write just a few tests, maybe there's only a 5% chance.  If I put a  
lot of thinking into my tests, I can get that percentage much closer  
to zero, but I know if I really want the holy grail of zero defects, I  
need help from mathematicians and formal methods.  Then again, quantum  
physicists may tell me the slight chance of improbable events means I  
can never get to zero.

Regarding advice for your friend, I can think of plenty of times when  
I worked at companies, people wrote a lot of code in a hurry, they  
wrote with different mind sets, they didn't write tests, and then the  
processing of tracking down flaws and bugs consumed painful months.   
It's a world of hurt.


Geoff

On Aug 23, 2008, at 23:45, Grant Rettke wrote:

> Hi folks,
>
> I recently had a good chat with my friend about unit testing. His
> company is thinking of making unit testing part of the "standard
> process". I gave him my take on unit testing, that:
>
> 1. Most developers don't know why they exist.
> 2. They are usually not maintained, or only maintained to gain code  
> coverage.
> 3. Project failure then "proves" that unit tests are pointless
> 4. Unit tests go away
>
> I suggested that this maybe combated with a "theory for development",
> primarily that there is some contract that comes from requirements and
> naturally translates into something that may be executed, namely a
> unit tests. This seems to be the critical link that %80 of developers
> haven't got, and that is why unit tests aren't written or maintained.
> Is it a simple idea? It seems that idea. I just want developers to
> have a reason for writing unit tests, a reason that they can
> understand and argue for or against. Such an approach seems like it is
> one of the seedlings for developing an overall theory for the
> development process.
>
> Is this what HTDP does, and leads to? I suspect it does. I will find  
> out.
>
> Best wishes,
>
> Grant
> _________________________________________________
>  For list-related administrative tasks:
>  http://list.cs.brown.edu/mailman/listinfo/plt-scheme