[racket-dev] PLTCOMPILEDROOTS

In v5.3.0.24, the new `current-compiled-file-roots' parameter and
associated PLTCOMPILEDROOTS environment variable give you more control
over where bytecode files are written.

The new parameter holds a list of paths. When looking for a compiled
file, the default `load/use-compiled' handler checks the paths in
order, where each path can be either

 * an absolute path, in which case the source file's directory is added
   onto that path (i.e., a re-rooting of the path), or

 * a relative path, in which case it's added to the source path's
   directory.

A sub-path from `use-compiled-file-paths' is then added onto that path
to find a compiled file, if any.

Tools that create bytecode files, such as `raco make', use the first
path in `current-compiled-file-roots' as the destination for new
bytecode.

The default value of the parameter is `(list 'same)', which means that
compiled files are written and found in the source file's directory
(under "compiled", etc.), as usual.

The PLTCOMPILEDROOTS environment variable is parsed in a similar way to
PLTCOLLECTS, but any literal "@(version)" in PLTCOMPILEDROOTS is
replaced with the Racket version number.


The Intended Use: Separating Source and Compiled Files
------------------------------------------------------

To put new bytecode files in "/tmp/compiled" instead of the source
tree, set PLTCOMPILEDROOTS to "/tmp/compiled:". The empty path
after ":" ensures that bytecode files are found at the usual location
when they already exist there and not in "/tmp/compiled".

Then,

 raco make /home/mflatt/x.rkt

creates the bytecode file

 /tmp/compiled/home/mflatt/compiled/x_rkt.zo

instead of

 /home/mflatt/compiled/x_rkt.zo

I implemented PLTCOMPILEDROOTS in response to a request for this
functionality.

The re-rooting function to get from "/home/mflatt" to
"/tmp/compiled/home/mflatt" is provided by `racket/base' as
`reroot-path'. It works the way you'd expect.


Another Possible Use: Versions
------------------------------

Since bytecode files are version-specific, using multiple version of
Racket on the same source files can create trouble.

If you set PLTCOMPILEDROOTS to "compiled/v@(version):", then the
bytecode files for different version coexist in separate sudirectories
under "compiled". For example, "x.rkt" compiled for v5.3.0.24 would go
in "compiled/v5.3.0.24/compiled/x_rkt.zo", while bytecode for v5.3.1
would (one day) go in "compiled/v5.3.1/compiled/x_rkt.zo".


Another Parameter?
------------------

You may wonder why I didn't generalize `use-compiled-file-paths', since
it seems to play a similar role. That parameter is where the "compiled"
sub-directory name comes from, and it's how DrRacket arranges for
bytecode files to be written and found in "compiled/drracket/errortrace".

The DrRacket example illustrates why I left the old parameter alone.
The `use-compiled-file-paths' parameter has become used to select among
different compilation modes, while the new parameter selects along the
orthogonal dimension of where files should be stored. In particular,
setting PLTCOMPILEDROOTS can make DrRacket write its bytecode files in
a tree apart from source files.

I don't like new search paths, and I especially don't like combinations
of search paths, but it seems to be the right thing this time.


What Python Does
----------------

See

  http://www.python.org/dev/peps/pep-0304/       [withdrawn]
  http://www.python.org/dev/peps/pep-3147/       [in Python 3.2.3]

The withdrawn PEP 304 proposal is close to PLTCOMPILEDROOTS but
effectively constrained to a two-path list, where the first one is
absolute and the second is empty. As the PEP 3147 discussion notes,
it's not clear why PEP 304 was withdrawn. Some drawbacks noted in PEP
3147 make sense, but I don't get the `__file__' problem. (It seems like
`__file__' might mean the source path or might mean the compiled file,
which seems unhelpful.)

The implemented PEP 3147 corresponds to a default PLTCOMPILEDROOTS that
is analogous to the versioning use above. I have long resisted this
default, because I worry about worsening the version clutter that
already happens in the add-ons directory. (My "~/Library/Racket" is
currently cluttered with 26 version directories, for example.) But if
you like that idea, now you can have it for the price of one
environment-variable configuration.