[plt-dev] New forms for requiring files -- poll for opinions and names

An issue that came up recently with David -- and one that comes up
every once in a while, is that of some project management tools.  A
summary of the problem is: you have some layout of files in your
project, and you want to be able to access them using in some symbolic
way.  Using strings as relative names works to a limited degree: it
means that the hierarchy is inflexible -- changing it requires
changing files too.  Also, the paths depend on both the requiring
module and the required one, usually you'll need a number of "../"s
that depends on the requiring file, and a then the path of the
required file.

Now that we have require/provide forms, it is possible to solve this
problem, and in typed-scheme Sam did something very similar to that,
which makes it a nice use case.  Here's an example from one of Sam's
files:

  (require (except-in "../utils/utils.ss" extend))
  (require (types convenience utils union subtype)
           (rep type-rep)
           (utils tc-utils)
           "signatures.ss" "constraint-structs.ss"
           scheme/match)

When this came up with David, I pointed at all the obvious places that
can be used to make it work, and then I continued to see if I can come
up with some way to provide more convenient ways to dealing with such
issues.  What I came up with is two require forms that I think would
do this job nicely:

* (file-in <expr>) -- this is close to `file', except that <expr> is
  an arbitrary expression (evaluated at the syntax phase, of course).
  It is similar to `filtered-in' (and uses the same
  `scheme/private/at-syntax' hack) in that you write some code.  Using
  the typed scheme snip as an example -- it would make writing those
  `types' and `rep' forms easy.  (With a minor point: as they are in
  this example, they would need to be macros, since their arguments
  are not quoted.)

  Also, it would be possible to have functions that consult some
  "configuration file" which defines the project layout (with the
  trivial case of the configuration file being some scheme module that
  is required for-syntax), require files relative to your home
  directory, the contents of an environment variable, the desktop
  directory, etc, and it could even do some cheap networking thing
  like downloading a file and then requiring it.

  In other words, it does a job similar to Sam's `define-requirer',
  except that it does so more generally, since you're using functions.
  It's a little more verbose since you need the `file-in' wrapper --
  but the advantage is that plain code is easier to write and would be
  readable to more people.  (For example, if you're looking at some
  random file in typed scheme, you won't know what it's supposed to
  do.)

This solves one side of the problem -- organizing code with such a
symbolic approach is becoming much easier.  But the other side of this
problem is that you'd want to centralize such code, and you need to
reach that central point conveniently from everywhere in your project.
Going back to the typed scheme snippet, this is the
"../utils/utils.ss" part.  This string still depends on the location
of the central configuration file wrt the project root.  I'm not sure
what would be the best solution -- the best thing I can think of is:

* (file-up <string>) -- searches for a path in this directory, then
  going up.  If there was some `or-in' form, then (file-up "foo/z.ss")
  this would be similar to:

    (or-in "foo/z.ss" "../foo/z.ss" "../../foo/z.ss" ...etc...)

  With this, Sam's code could use (file-up "utils/utils.ss").

Combining these two, and assuming that they're provided by
`scheme/require', a typical "project management" code chunk could look
like:

  (require scheme/require
           (file-up "config.ss")
           (file-in the-foo-utility
                    (subsystem1 'blah)
                    (subsystem2 'sheep/goes/meh)))

where the "config.ss" file provides (for syntax) the definitions for
the subsystem functions and the first value.  (One tricky bit: the
order of the three require clauses is important.)

One point that Matthew raised when I talked to him about this is that
it can lead to a mess if the functions that you're using in `file-in'
are non-deterministic.  This problem is already in now, of course, the
only thing that changes is how easy you can get to it.  But given the
utility of these forms (in contexts that make this pop up every once
in a while), I think it should generally be fine -- as long as the
documentation has the right warnings, as well as some boilerplate code
that most people will just copy and modify.

So, are there any opinions on this?  Or on the specific forms?

Also, I'm not sure about the names.  The `file-in' vs `file' (vs
`file-up') seems like it can be confusing, so maybe `path-in' and
`path-up' would work better?  Another alternative is to have a
*function* that does the up-search, and provide it for syntax, with a
use-case like:

  (require scheme/require
           (file-in (look-up "config.ss"))
           (file-in the-foo-utility
                    (subsystem1 'blah)
                    (subsystem2 'sheep/goes/meh)))

but this seems like it can be much more confusing.  Another option is
some `file-in-up' which combines the two features (expects an
expression, and does the search with the result) -- this seems to me
like cramping too much functionality into a single tool.

Yet another option is have `file-in' be some `#%app'-like thing, so
the above code becomes:

  (require scheme/require
           (file-up "config.ss")
           (file-in the-foo-utility)
           (file-in subsystem1 'blah)
           (file-in subsystem2 'sheep/goes/meh))

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!