[racket-dev] Splitting the `pkg` implementation

As part of making the "core" of Racket smaller, I'd like to propose
separating out part of the package system implementation. In
particular, I'd like to make the core portion of the package
collection not use the network. This would allow us to remove the
`net` and `json` collections, something like 7500 lines of code, from
the core. [1]

I think, from looking at the code, that there are two major ways the
network is used right now: (1) fetching packages named at remote
locations, and (2) fetching the contents of the remote catalog.

For (1), I think this should be made extensible.  There should be an
info.rkt field, named say `pkg-installer`, which specifies [2] a
function like `package-source->name+type` and also a function like
`stage-package/info` as well as a type. Then the relevant functions
use `find-relvant-directories` to find all the extensions, and call
them in order to see if they match. This would both allow parts of the
current implementation to be taken out of the core (such as the
GitHub-specific code) but allow people to extend the package manager
with new ways of specifying packages, such as with git urls directly.

For (2), the case is a littler harder. All the places I can find that
use the network in this case are inside `catalog-dispatch`, and use
`read-from-server` to talk to the network. I propose that
`catalog-dispatch` pass `read-from-server` as an extra argument to the
server callback, and only call the server callback if the `net/url`
library is available.  I'm not entirely happy with this test, though.

Finally, there's a lot of url manipulation that doesn't use the
network.  I propose to move the portion of `net/url` that does this to
a separate `net/url-struct` library, which would be re-provided by
`net/url`.

Sam

[1] We could move (and I plan to) most of this either way -- the core
doesn't depend on `net/imap`.
[2] Probably by specifying a module path that would provide a few known names.