[racket-dev] PLaneT(2): Single vs multi-collection packages

I've been hesitant to comment on any of this, for three reasons: (1) 
I've read the new package system documentation on at least 3 separate 
occasions, and -- perhaps because I'm biased by having already formed 
some ideas about where I'd like things to go -- I've had trouble 
understanding the rationale behind a lot of the decisions; (2) I'm not 
familiar enough with DrRacket internals to know whether there's some 
unusual barrier to keeping things simpler than it seems like people have 
been talking about, so I can't help with that; and (3) I'm confused 
about what's already decided and what's open to discussion, and suspect 
it would take many more hours to understand.

Since I'm heavily invested in Racket packages but lost in the current 
discussion (sorry), all I can offer right now are some comments that 
start from the concepts of the original PLaneT, in case anyone who 
understands the new system sees anything of interest in this quick brain 
dump...

For all of my packages, as well as any package I can imagine, I think 
that the original PLaneT got many things right or close to right:

* Some kind of unique package-name controlled by a developer.

* Multiple versions for a given package-name (I'll call them 
package-versions in this email) can be installed, and there is some 
version selection mechanism.

* Files from a package-version end up grouped together in the directory 
structure, specific to that package-version, and certainly not mixed 
into directories with files from package-versions of different 
package-names.

* Flat namespace (let's ignore the PLaneT package-owner part for now), 
without attempt to name packages according to some topical ontology.

* Metadata in "info.rkt".

I was expecting to use this original PLaneT as a starting point, and 
evolve it in ways like the following...

* In addition to the "(planet ...)" require-specs, package-versions also 
can come from "http:", "https:", and "git:" URLs.  ("github:" would also 
be OK.)  Each such URL would identify trees or a tarball.  Then we see 
how people choose the PLaneT server vs. HTTP vs. Git over time.

* Simple web-of-trust package-version public-key signing of 
package-versions (e.g., URLs plus hashes of contents), to start with, 
perhaps initially with only centralized repository for signatures.  Soon 
build distributed web-of-trust, plus multiple repositories so 
organizations have option to keep their signatures separate.  Build 
mechanisms atop that, including advancing the state of the art.

* Maybe improve the version-selection and compatibility support.  
Investigate whether it's worthwhile to separate out the 
backward-compatibility information from the static package-version 
distribution (and especially from the version number), or whether in 
practice there are simpler ways that are satisfactory.

* Maybe a facility in "info.rkt" to provide aliases for require specs.  
Otherwise, people writing nontrivial multi-file code that uses other 
packages from PLaneT/whatever end up having to make wrapper modules so 
that we don't goof our require-specs and accidentally depend on multiple 
package-versions for the same package-name.  Note that, with URLs, these 
aliases *might* be the only actual package-name construct in the 
HTTP/Git system as distinct from URL similarities of package-versions.  
This info might be implicit in a package-version's "info.rkt"'s 
reference to a previous package-version, perhaps coming from an 
assertion of compatibility info.  This might be simpler than it might 
sound, but it has some interesting implications, including for forking 
and web-of-trust.)

* Automate and simplify releasing in general.  With PLaneT, it's been 
not-unusual for even core Racket developers to avoid releasing some 
add-on code to PLaneT, perhaps because the clerical stuff was a 
headache.  For the old PLaneT, I was simplifying this with McFly, but 
with new a package mechanism, I would start with that and then ask what 
clerical parts still need help.  (For example, if doing development in 
an SCM repository that's accessed directly via require-specs, then 
releasing a package-version might consist mainly of adding a tag/label.  
planet-lang.org's directory might even update automatically, given info 
about a previously-released package-version of the same package-name.)

* Use submodule support to support single-file packages, at least for 
the HTTP/Git package-versions.  "(module+ info ...)".  It seems from 
Emacs history that some people really like the single-file module, it 
lowers barriers, and now submodules give us an easy way to finally do it.

* Do whatever is necessary to avoid blocking the program for 
few/several-minutes while documentation is reformatted, when requiring 
an uncached package-version.  Maybe even moving it to an async process 
that's run when idle (Unix "nice"?) would work.

* To put it vaguely: keep things simple in most cases, but don't 
dumb-down in practically restrictive ways, and keep an eye out for 
places to experiment with potential big wins for immediate practice or 
research.  Some things I just mentioned above would surely need 
refinement/exclusion based on this principle.  For another example, I 
heard some comments at one point about a package name being an 
interface, and multiple sources being able to provide implementations 
matching that interface.  I don't know the current plans for that, but I 
wouldn't make any special mechanism for that.  For another example, 
don't try to dumb-down package-names, as if the first person to make a 
package concerning the generic concept "foo" has the be-all-end-all 
package for all things "foo".

* Some Web directory of software on "planet-lang.org" (with JSON dump 
and maybe query), which includes both PLaneT packages together with the 
HTTP/Git packages that people have chosen to list in the directory.  
It's a "this is all the Racket packages we know about, and probably 
easier to find via search here than via Google."  (Eventually, this 
would be hooked up to the site-wide search feature for 
"planet-lang.org", together with other categories of other searchable 
Racket-related info that we identify.  Then DrRacket search could be 
hooked up to that.)

Neil V.