[plt-scheme] [maybe off-topic] The theory of testing

In my opinion, HtDP is one of the few books that do integrate specification, 
testing and coding. Yet I think that for a substantial software project even 
more has to be considered:

1: What are we aiming for? (a questions whose answer usually is not clear at 
the very beginning, but should be made clear during the early stages of the 
development)
2: A preliminary answer to question 1 (but not yet a complete specification)
3: Feasibility studies: do we have the apropriate algorithms and resources? 
Are they sufficient to do the job within constraints of space and time?
4: Repeat 1, 2 and 3 until something like a specification is obtained.
5: Specification includes:
5a: What will the user see: a user manual (in my opinion to be produced 
before going on to the steps to follow) The user manual can also be a test 
as to whether or not the product will satisfy the expectations of the user. 
Let the user judge. If necessary repeat from step 1.
5b: A design (programmers manual): what the programmer has to produce, what 
languages and software will be required.
5c: An essential part of the design is a testing suite (both correctness and 
measurements of required space and time)
5d: Tests must include normal use but also all situations at the very edge 
of the specification (where apropriate, bulk tests included; this may 
require the preparation of large data-sets, which have to be designed and 
produced too)
6: Although at this point we dont have any working code yet, the project is 
almost finished. Your boss may be nervous about the fact that there is 
nothing to show working yet, though.
7: Do the coding or have it done by programmers (programming is not the same 
as designing)
8: During coding of the product proper and its tests, apply the tests 
immediately as far as possible. If the project is well designed, tests can 
be applied to separate parts of the project.
9: Deliver: Finished (apart from maintenance)

I followed this procedure for a major file-system project (some decades ago) 
I left the project during step 6 (my boss very stressed), but three months 
later the product was delivered (the person who did steps 7, 8 and 9, is 
stil a good friend of mine). As far as I know only once a bug was found 
after delivery (found by myself, identified within 5 minutes and fixed (+ 
tested and redelivered) within a day) In the above scheme I did not consider 
the aspects of costs and deadlines. These aspects may put a great stress on 
the correct evolvement of the process (I got engaged in the project after it 
was behind schedule by 400% already even without finishing step 1 properly. 
Some prestige was involved, therefore costs were not a main issue. The 
original deadline was one year. I took 9 months for steps 1-6. The remaining 
steps took 3 months: total 1 year when not counting the time wasted before I 
got into the project)
Jos

----- Original Message ----- 
From: "Grant Rettke" <grettke at acm.org>
To: "plt-scheme List" <plt-scheme at list.cs.brown.edu>
Sent: Sunday, August 24, 2008 5:45 AM
Subject: [plt-scheme] [maybe off-topic] The theory of testing


> Hi folks,
>
> I recently had a good chat with my friend about unit testing. His
> company is thinking of making unit testing part of the "standard
> process". I gave him my take on unit testing, that:
>
> 1. Most developers don't know why they exist.
> 2. They are usually not maintained, or only maintained to gain code 
> coverage.
> 3. Project failure then "proves" that unit tests are pointless
> 4. Unit tests go away
>
> I suggested that this maybe combated with a "theory for development",
> primarily that there is some contract that comes from requirements and
> naturally translates into something that may be executed, namely a
> unit tests. This seems to be the critical link that %80 of developers
> haven't got, and that is why unit tests aren't written or maintained.
> Is it a simple idea? It seems that idea. I just want developers to
> have a reason for writing unit tests, a reason that they can
> understand and argue for or against. Such an approach seems like it is
> one of the seedlings for developing an overall theory for the
> development process.
>
> Is this what HTDP does, and leads to? I suspect it does. I will find out.
>
> Best wishes,
>
> Grant
> _________________________________________________
>  For list-related administrative tasks:
>  http://list.cs.brown.edu/mailman/listinfo/plt-scheme