[racket] Planet2 questions

At Tue, 9 Apr 2013 18:18:35 +0200, Berthold Bäuml wrote:
> I hope to find time to write up the model this weekend. But what is
> absolutely clear to us is that a package management system has to
> have a notion of versions (or variants as I called them) for a
> package at the lowest level -- otherwise such simple things as
> roll-backs or composing a system from packages in a deterministic way
> (i.e., by precisely specifying a version for all packages) is not
> possible. Of course, on top of such a low level system one can add
> more convenient interfaces for users for the more standard use cases,
> e.g., mimicking the version disregarding behavior of planet2 -- but
> it is not possible to do it the other way round.

I have been working with Jay on the package manager recently, and so I
have some further info on this point.

Building versions into the lowest level could mean different things:

 1. It could mean that a packages are referenced in programs with a
    specific version number.

 2. It could mean that the package system has a built-in way to declare
    dependencies not only on a particular package, but on a particular
    revision of a package --- either to ensure that certain features
    are available or that certain bugs are not available.

 3. It could mean that there's a fine-grained way to say which
    implementation of a package should be used in a given
    installation/configuration.

#1 was a prominent feature of Planet1 that we've dropped in the new
package manager. In fact, packages are not referenced at all in program
sources; only collections are referenced, and some collections turn out
to be supplied by packages in a given installation/configuration. That
is, package management and program execution are more separate.

#2 is built into the new package system, though in an intentionally
simplified form compared to Planet1. All version X specifications in
dependencies mean "at least version X", and those specifications are
intended to ensure the availability of new functionality or fixes that
were added in a backwards-compatible way. Backwards incompatibility is
handled by creating a new package name.

#3 is at the core of the new package system, and the rest of this
message is about that one. I think it's probably close to what you have
in mind; I'm interested to hear more about whether you mean something
different.

----------------------------------------

To be clear about #3, we have to distinguish "package name" from
"package implementation". A package name is something like "mischief",
which you use for installing and declaring dependencies. A package
implementation is something that you download from, say,

 https://github.com/carl-eastlund/mischief/tarball/     fe7119517a4dcd3f5c509735a7b5a5664151c14f

Note that a package implementation in this sense corresponds to
specific revision of a pile of code, such as a particular commit in a
git repository. The package manager includes the concepts of a "package
source" and a "checksum", which together tell you how to get a package
implementation. (That implementation may have its own version number,
which corresponds to #2 above, but such a version number is in
principle orthogonal to the package implementation's checksum.)

The mapping from a package name to a package implementation is provided
by a "catalog". (This is recent terminology; until last week, a
"catalog" was variously called a "package name resolver" or "index".)
PLT provides a catalog server at pkg.racket-lang.org, but you can make
your catalog (as a server or on a local filesystem), and so you can
precisely control the mapping from package names to packages.

Furthermore, we've recently added tools to `raco pkg' to make it easier
to manage catalogs. For example, if you want to take a snapshot of the
current pkg.racket-lang.org and use that from now on (so that the
mapping from package names to packages doesn't change), use these
commands:

 raco pkg catalog-copy  https://pkg.racket-lang.org /full/path/to/catalog/
 raco pkg config --set catalogs file:///full/path/to/catalog/

You can modify the files generated at "/full/path/to/catalog/" by hand
in a fairly obvious way. Or you can upload the directory to a
file-serving HTTP site and point installations to the uploaded
directory as the catalog. There's also an option to use an SQLite
database as the format for a catalog, which is a better option if you
want to modify the catalog programmatically via `pkg/db', but an SQLite
database is less easy to use from a file-serving HTTP site.

In particular, I can imagine having a project whose source code
includes a package catalog. To upgrade a particular package, I'd change
the catalog and `raco pkg update'. When I commit a particular revision
of the source code to a git repository, the package catalog is saved;
then, I can roll pack the project (including its references to specific
package implementations) to any previous version with its associated
package implementation via a `git checkout' (or whatever) plus `raco
pkg update'. Working this way, the package catalog acts a lot like git
submodules.