[plt-scheme] Re: Turnkey was Re: search paths and teachpacks?? was Re: [plt-scheme] How do I get picture.plt working under DrScheme 205?

On Thu, 25 Sep 2003, Joe Marshall wrote:

> It is hard to make something foolproof because fools are so ingenious.
>
> I was once a QA engineer at a dysfunctional company.  I used to write
> tests that would attempt to exercise the edge cases.  I'd pass empty
> strings as pathnames, ".." as file extensions, attempt to delete
> databases that didn't exist, refer to the local machine with a fully
> qualified network pathname, and other such idiotic behavior.

Yes, I've done this sort of testing too. It is useful, but not
comprehensive.

The opinion that I am presently forming is that you should refactor your
system moving as much of it into libraries as you possibly can and then
test those libraries as separate entities in a white-box fashion.

This sort of sugestion would likely have been quite unpopular at any of
the companies I QA'd for because it would mean "touching" lots of code.
Developers are afraid of changing their large systems for fear of making
that one little change that brings down the whole house of cards.

>
> For some reason I cannot comprehend, this kind of testing was
> considered `wasteful' because it didn't test `the product under normal
> use'.  The engineers were annoyed because they had to insert checks
> for things that `nobody would ever do'.
>

This kind of testing attempts to exercise code paths that are hard to
reach under "normal" conditions. If on the other hand we had refactored as
described above, then we can systematically test each code path in a
modular fashion.

> I eventually left that company (as did most of the QA department).

Yup. Software QA in industry is a tedious, boring and unsatisfying
ocupation.


> To this day, I still think this is the right thing to do.  If the
> software can stand up to deliberate idiocy, it is more likely to
> behave correctly when you start using it for things that were
> unforseen by the implementor, for example, when you attempt to attach
> it to other software via some sort of scripting.

Imagine how much easier such scripting could be if much of the code were
already factored out into libraries.