[racket-dev] experiment reorganizing the repo into packages

[shuffled]

20 minutes ago, Matthew Flatt wrote:
> Just to be clear, I agree that we want to minimize cycles.
> 
> I don't think that actually prohibiting cycles is going to be workable,
> though, especially at the level of documentation.
> 
> Cycles at the level of "-lib" suggest to me that the organization
> should be improved.
> 
> Cycles at the level of "-doc" suggest to me that the documentation is
> thorough and helpful.

I completely agree with the first two paragraphs, and with the two
conclusions.  Specifically, it seems to me that allowing all cycles
first, discouraging code dependencies next, and trying to get to no
code cycles last and possibly finally prohibiting them is a very
practical approach to dealing with the dependency mess.  (And here
"code" is source code except documentation source.)

Trying to do the right thing now is probably hard enough to make it
not actually happen, making it impractical.  I'm also very happy (and
positively surprised) to hear you say that most -lib packages are not
cyclic.


30 minutes ago, Matthew Flatt wrote:
> [You had *lots* of comments about specific packages --- which are
>  certainly welcome, but let's tackle the big picture, first.]

Absolutely.  Like I said, much of it is a laundry list to keep in some
todo list, which I did in an attempt to help organizing it and since
I'm instinctively looking for these kind of issues...

(BTW, I see now another possible problem: both the xrepl packages have
an info file, with a scribblings entry in one of them.)


Now, about this higher level issue:

> > * I usually like the fine split to packages like -docs and -lib.
> >   However, minor subpoints:
> > 
> >   - Shouldn't the "-docs" suffix be "-doc" to be in-line with the
> >     directory name and also symmetric to "-lib"?  (BTW, looking at
> >     the Fedora installations that I have, there are both
> >     conventions, but "-doc" is much more common than "-docs" , and
> >     "-libs" is a little more common than "-lib".)

(This is a technicality.)


> >   - I think that a -lib package is for cases where there is some
> >     use for it by itself, and in some cases there's no need for
> >     that.  Things that I've seen:
> >     - "xrepl-lib" -- there's no library functionality in xrepl
> >       that is useful by itself, and if there is (or if something
> >       does become useful), then it should move out of it.
> >     - "at-exp-lib" -- the "lib" in the name seems confusing
> >       (because there is no "at-exp", and I don't think that it's
> >       right to have a "-lib"-only package...)
> >     But I was confused:

These items make sense after I saw the rest of it -- and pending a
nicer solution (to avoid the multiple repositories/directories thing).


> > [..."package distribution kinds"...]
> 
> Well, I agree with all these thoughts, but what's the conclusion?
> 
> There's no requirement that packages be in multiple repositories,
> but different packages in a single repository currently have to be
> different subdirectories in the repository. Do we need (or do we
> really want) a notion of "subpackages" to support some general form
> of splitting within a package?

I think so.  I have several reasons for this.  First, I don't see
these kinds as a closed set -- it's easy to imagine new kinds of
distributions (devel vs application, compiled & optimized vs
debuggable, textual/command-line vs gui, drracket tool vs emacs or vim
plugin, etc etc).  Worse, it's easy to imagine such kinds that are
applicable in contexts that are specific to just a few or even one
package (like the UI language for the string constants or for a
language analysis tool, or different foo themes for the foo
application).

Second, given that they're open-ended, I can see people revising their
packages by adding such kinds later.  An easy example: take a random
collection, and add a small TR interface -- probably no change --
extend it to have the full functionality available for TR, and that
will cause changes.  Similarly, make lazy into a more serious
language, and now many packages will want such changes.  And even if
it's a directory-per-kind, the thought of moving files around because
of the different distribution modes seems wrong.

The first thing makes me want to have user-definable distribution
kinds.  The second thing makes me want to do that via a level of
indirection.  Taking both, I'd like to see something like this:

  (define name "eXtended REPL")
  (define distributions
    '([lib "*.rkt"]
      [doc "*.scrbl" "scribblings"]
      [he  "hebrew-commands"]
      [binary-debugging "libs/portable-gdb.so"]))

And I'd like that to be used for distribution: either to create
packages (of different kinds), or more easily, pack up the whole thing
and have the client decide which parts to actually install (as with
some windows installers).

There's a bunch of technical questions here.  Like the specification
of file subsets to use: filters, unions/intersections, etc.  Given
that you have a simple split, this can start very inflexible, and add
things as needed later on -- so avoid getting some super flexible
thing (and fall into the dist-spec pit).  Another problem is
specifying dependencies, which can be in a natural way:

  (define deps
    '([lib ("readline" lib) ("scribble" lib) ("macro-debugger" text lib)]
      [he "hebrew-dictionary"]
      ...))

There's more missing technicalities, but I think (=> hope) that
something along these lines can work.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!