[plt-scheme] Unix install

At Wed, 3 May 2006 00:42:45 +0300, Lauri Alanko wrote:
> On Tue, May 02, 2006 at 07:27:47AM -0400, Eli Barzilay wrote:
> > On May  2, Lauri Alanko wrote:
> > > I had always assumed that the file organization and the package
> > > format were deliberate design decisions rather than acknowledged
> > > deficiencies.
> > 
> > Like Matthew said, the file organization is friendlier to everything
> > in a single directory which is the only layout on Windows, and it's
> > working on Unix too.
> 
> Ah, right. It wasn't considered worthwhile to customize the file
> hierarchy for various platforms individually, so a single one that works
> somewhat reasonably on all platforms was selected.

I don't understand what you mean.

Here are two of our requirements:

 1. There should be one format for all platforms, to ease the burden on
    package producers and consumers (in the common case that a single
    set of files works on all platforms).

    For example, I don't want to create multiple packages for a
    teachpack that I distribute for assignment #17 in my class, and I
    don't want to students to download --- and attempt to install ---
    the wrong package for their platforms.

 2. The format must be robust against various encoding changes.

    Downloading "binary" files in "text" mode is less of a problem than
    it used to be, but students are fairly creative in the
    byte-mangling ways that they manipulate files.

> > First of all, the current format is based on existing tools: base64
> > and gzip.  It is possible to edit it as quack does.  Below that there
> > is a convenient to edit format for the contents.  What's missing?
> 
> Just minor points: having the whole file be plain base64 text without
> any headers means that it cannot be recognized as a package of any sort.

Making the file recognizable sounds like a good additional requirement.
I agree that we missed it last time.

> Besides, base64 is at times a useful encoding for transporting over
> unreliable networks, but it makes no sense specifying it as an integral
> part of the file format.

As you see above, we imagined that transport over unreliable networks
(i.e., programs used by less-than-clueful students) would be the norm.

> The .plt format conflates three distinct
> concepts: the actual organization of the data, compression, and ASCII
> armoring. And instead of being named .plt.gz.b64, it's just ".plt", to
> further obscure the structure of the data.

When we picked the naming convention, we still had to live in the 8.3
universe of the DOS filesystem --- a requirement that wouldn't occur to
me now, and I didn't think of listing above. It's certainly not a
requirement for the next round.

> And although the actual internal data format (the one that starts with
> magic "PLT") is not too bad, it just seems like reinventing the wheel:
> there are already a gazillion ways to pack multiple files into a single
> one. Granted, being purely text-based is a bonus, or would be if the
> content wasn't afterwards scrambled with gzip and base64.

I agree. Honestly, the random format was a shortcut, because I was too
lazy to learn about an existing format. (And I had so much free time
back then, too!)

> Finally, the only "official" tool for manipulating .plt -files is
> setup-plt, which doesn't really offer very many choices: the only thing
> you can do with a package is install it. Unlike an "ordinary" package,
> there is no way to just list its contents, or extract it into a
> temporary directory and see what's inside it.

The format is documented, and always has been.


Here's a suggestion:

 * Ask us about the requirements that we know

 * Add the requirements that you know and we don't

 * Design a format and build tools that work better.

Then we'll all be happier.


> > > However, since the usual way is to compile everything once and for all
> > > during installation, then .../share is probably the right place for the
> > > .zo files. Especially so if .zo files ever get installed alone,
> > > _without_ the source.
> > 
> > We had some talking about this.  Some of the `compiled' directories
> > have native subdirs (sorted by platform), and there are even a few (1
> > currently) .zo files that are platform dependant.
> 
> Wow. How does this happen? Some FFI macros expand differently on
> different platforms?

No. In the "sgl" collection, "gl-info.zo" is generated at setup time
with platform-specific constants. Even though it's platform-specific,
putting it in a ".zo" file (as opposed to some file in "plt/lib") means
that things will automatically work right when someone creates a
"stand-alone" executable.

> This relates to a point I raised a while ago: how should expansion-time
> variance be handled?

For now, we blame the programmer who wrote code that expands variantly.
But "gl-info.zo" is an exception for now. Also, the soon-to-appear
library that describes the directory configuration will be
installation-specific.

> > The conclusion of the discussion was that it would be easy and
> > acceptable to have all collects in .../lib/plt (Ari mentioned that
> > this is already done by python, perl, and ruby).
> [...]
> Still, it is technically wrong.
> [...]
> Still, I realize that conforming to this would require quite a bit of
> effort, so it may make sense to defer it to unforseeable future.

Sounds like we're on the same page, then.

Matthew