[racket-dev] experiment reorganizing the repo into packages

On Thursday, Matthew Flatt wrote:
> 
> You've sketched out the producer side, and I'm not sure of some
> about some of those details.  The consumer side seems even more
> complex to me.  It seems like the package system would have to keep
> track of which subpackages are installed for a package, provide an
> interface to the user (e.g., in the GUI) for subpackages, and be
> able to update a package with new subpackages --- all while tracking
> dependencies at the level of subpackages. I'm sure it can all be
> done, but I'm not sure how difficult or important it will be
> relative to everything else that still needs to be done, and I'm
> pretty sure it can't all be done right now.

A very cheap way to do this is to use these sub-package specifications
only for creating packages for distribution.  With the obvious
resulting package file names, this means that there is no change at
all that is needed on the consumer side.  And I think that while it
looks like it reduces the whole thing considerably, it's still worth
it for some important advantages that are still kept:

* No need to move files around in a distribution-oriented layout in
  code repositories (which is important since people really maintain
  the freedom of puting files in a way that fits their needs
  logically).

* No restriction on a few known "-foo" names -- keeping it open for
  new kinds of "package kinds".  (Related note: I spoke to Ryan about
  having db in the minimal core and the possible reason for this being
  due to sqlite -- it would be nice, and according to Ryan not too
  difficult, to split the db package into several "db-foo" backends,
  which hooks back into my point of some splittings that make sense in
  just one package.)

* Minimize code hacks that compensate for the lack of such
  sub-packages.  There will still be some hacks -- for example, it
  will be convenient in the beginning to just show available package
  names and the end user will need to figure out what they mean, and
  when later on you want to grouping back packages that really came
  from a single source for the UI, instead of some hack that is based
  on names, it can use this information instead.

All of these are important IMO, but the last one is particularly good,
since hacking something based on naming conventions means that these
hacks will stay around for a very long time -- which will lead to yet
more hacks.  In the spirit of making things usable, I also worry about
accumulation of command-line flags, a problem that plagued .plt file
generation and installation (making it into black magic that not many
people could handle).


> Instead of trying to create subpackages, we might stick with a
> naming convention. Even with a naming convention, it might make
> sense to extend the package-manager GUI so that it recognizes the
> convention and groups related packages together in a list view (but
> all the machinery remains at the package level).

(This is an example for the kind of hacks where some of them can be
avoided with the above, and with others I think that it will be easier
to "recover" proper code after a transition period with such hacks.)


> Also, a package implementor is free to develop "subpackages" along
> the lines you suggest, but splitting out the subpackages into
> packages before handling them to a catalog.

(And having scripts to do such splitting by developers would not be
needed at all.)


> Or, we could just not try to slice packages as finely for now, so
> that the "-lib" and "-docs" packages of the experimental split are
> merged together --- leaving a future split possibly to subpackages.
> That will most likely result in a coarse layering that is close to
> "build-deps" up to the "drracket" layer, but still provide more
> separation for things that are outside the layer needed by DrRacket
> itself; for example, "games", "redex", "htdp", "plai",
> "picturing-programs", and "plot" can still be separate packages, I
> think.

FWIW, I'd view this (avoid fine slicing) as a possible way to avoid
growing hacks, but I really like the idea of smaller packages and the
benefits it comes with.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!