[plt-dev] compiler/cm and .dep changes

From: Matthew Flatt (mflatt at cs.utah.edu)
Date: Fri Apr 30 22:21:52 EDT 2010

The compilation manager --- as implemented by `compiler/cm' and used by
tools such as `raco make', `raco setup', and DrRacket --- previously
based its decision to recompile a file using only timestamps. It now
uses SHA-1 hashes to reduce recompilation when timestamps change but
source-file content doesn't. (This is useful with various git
operations that change the timestamps on source files when the content
doesn't change.)

If a source file's timestamp changes but its SHA-1 hash does not, then
the bytecode version of the file is updated by simply setting its
timestamp. Similarly, if the bytecode file for a dependency changes,
but the bytecode's SHA-1 doesn't change and neither do any of its
further dependencies (recursively), then the dependent bytecode file's
timestamp can be adjusted instead of recompiling.

When you run a program, the load handler still chooses between source
and bytecode based just on the immediate timestamps. That's why the
compilation manager updates the bytecode file's timestamp when it
doesn't need to recompile from source.

If you change a file by just editing a comment, whitespace, or
something else that doesn't affect a module's meaning, then it's
possible that the generated bytecode will be the same, in which case no
file that depends on the changed file will need to be recompiled.
Compilation uses various `gensym'-like operations, however, so I'm not
sure how often the bytecode file will turn out to be the same.

The format of a ".dep" file has changed to record relevant hashes.
Specifically, a ".dep" file now records a SHA-1 hash for the source and
a hash that combines information for the compiled form of each
dependency. For each dependency, the combined hash includes the hash of
its bytecode plus the combined hash from all of the dependency's
dependencies as recorded in the dependency's ".dep" file.

A SHA-1 hash is computed through the OpenSSL library, which we already
use/include in our distribution.

Posted on the dev mailing list.