[plt-scheme] Unix install

On May  2, Lauri Alanko wrote:
> On Thu, Apr 27, 2006 at 08:44:47PM -0600, Matthew Flatt wrote:
> > We could use some help. Whether it's the filesystem layout, the package
> > format, or anything else that bothers you, please help us fix it.
> 
> I had always assumed that the file organization and the package
> format were deliberate design decisions rather than acknowledged
> deficiencies.

Like Matthew said, the file organization is friendlier to everything
in a single directory which is the only layout on Windows, and it's
working on Unix too.


> Certainly someone chose to design and implement the .plt file format
> instead of using existing archive formats (possibly with some
> additional metadata in a special file, like what jar does). So
> before something can be "fixed", there has to be some kind of a
> consensus that the current way is indeed bad and a change is in
> order. Certainly I'm not expecting everyone to share my opinions on
> what is the Right Thing.

First of all, the current format is based on existing tools: base64
and gzip.  It is possible to edit it as quack does.  Below that there
is a convenient to edit format for the contents.  What's missing?


> But, for what it's worth, here's how I think the PLT software
> _should_, in an ideal world, be installed on a "standard"
> (FHS-compliant) Un*xish filesystem hierarchy. Here "..." is stands
> for the installation prefix, typically /usr or /usr/local.

This would be the last step in the current batch of changes.  The
funny thing is that I already had a very similar document to this, to
plan for the change.  (It also went through the current Debian
packager (Ari Pollak) for comments.)  So your comments come at a very
convenient time.  Further comments below.  At the bottom I pasted the
whole thing that I currently have with Ari's comments on of yours.  If
you have any comments please mail me off-list.


> Binaries: .../bin/
> 
> At least all the "full applications" (drscheme, mzscheme, mzc,
> slatex, etc...) should go to .../bin where they are immediately in
> the user's path. If anything is considered an "auxiliary binary"
> that is not meant to be called directly by the user (help-desk,
> perhaps?), then it should be placed somewhere under .../lib/plt/.
> The same goes for framework-test, although tests probably should not
> be installed at all by default. In any case, names like "help-desk",
> "web-server" and "slideshow" are perhaps just a bit too generic for
> placing in a directory that is shared with other software. Maybe
> they should be prefixed with "plt-" or renamed altogether?

These are indeed the three problematic names.  I had the "plt-" prefix
thing, and added the alternative path idea now.  BTW, help-desk is
also a problematic name for the man page.


> Common native libraries: .../lib/
> 
> "Common" means those libraries that a user may link directly against
> when developing applications that embed mzscheme: libmzscheme, libmzgc,
> libmred. When placed in .../lib/, a linker can find them without extra
> path tuning.

This is an option, not clear if it will be easier or not than keeping
it in .../lib/plt/.


> I'm not sure what "mzdyn.o" is. I'm guessing that it should be under
> .../lib/plt. The same goes for "buildinfo".
> 
> 
> Common headers: .../include/plt
> 
> Again, "common" means anything that is needed for embedding.

Why differentiate embedding from other headers?  They should all be in
there.  (Most are used for both situations anyway.)

> If there are more than one or two headers for a library, it is good
> style to keep them in a separate subdirectory, and I think that the
> threshold is met here.
> 
> The mzc-specific headers are platform-independent (right?), so they can
> probably also be installed under .../include/plt.
> 
> 
> Man pages: .../man
> 
> Straightforward.

Modulo "help-desk.1".


> Scheme code: .../share/plt/
> 
> Scheme code is platform-independent and may be shared by multiple
> architectures, so it should be placed under .../share.
> 
> 
> Compiled scheme code: .../share/plt/
> 
> .zo files are a bit problematic. They are platform-independent, which
> would place them under .../share, but "feel" more like binaries than
> shared data, which would place them under .../lib. Also, they are
> redundant and can be regenerated from other files, which fits the
> description of /var/cache and /var/local/cache. Certainly, _if_ modules
> were compiled on the fly and the compiled versions were cached, then
> /var/cache would be the right place, although this kind of tweakery
> would require playing with permissions. This is what man and tex do.
> 
> However, since the usual way is to compile everything once and for all
> during installation, then .../share is probably the right place for the
> .zo files. Especially so if .zo files ever get installed alone,
> _without_ the source.

We had some talking about this.  Some of the `compiled' directories
have native subdirs (sorted by platform), and there are even a few (1
currently) .zo files that are platform dependant.  The conclusion of
the discussion was that it would be easy and acceptable to have all
collects in .../lib/plt (Ari mentioned that this is already done by
python, perl, and ruby).  So the current best solution is to have
.../lib/plt/collects with the same structure it currently has.


> Native libraries for collections: .../lib/plt/
> 
> These are platform-dependent and not used directly by the user. Hence
> they should be "hidden" under .../lib/plt. Perhaps there could be a
> similar collection hierarchy under .../lib/plt as there is under
> .../share/plt.
> 
> 
> Documantation: .../share/doc/plt

See Ari's comments -- he suggests .../share/plt/{doc,notes}.


> All the documentation should be in one place. Now some is in
> .../collects/doc and then there are doc.txt -files in all the
> collections (except mzlib).

This (moving doc.txt files) is not likely to happen, since it is part
of the collections.


> It is also a bit inconvenient that all the doc.txt -files have the
> same name: I often have a number of them open in emacs, and it's a
> bit hard to know which is which when all you have are buffers named
> doc.txt, doc.txt<2>, doc.txt<3> etc.

[
  (require 'uniquify)
  (setq uniquify-buffer-name-style 'post-forward)
  (setq uniquify-after-kill-buffer-p t)
]


> So perhaps the best way would be for the contents of current
> collects/doc to be in .../share/doc/plt, and then move each
> foo/doc.txt into .../share/doc/plt/collects/foo.txt.

(BTW, info.ss can be used to specify a different doc file name, which
is another factor in making this hard.)


> Also everything currently under "notes" should be moved here.

Yes, see above.


> That's about it. To me it seems that most of this can be
> accomplished just by changing installation paths a bit.

It's more than that -- it will require locating the pieces of code
that rely on things being in a particular place and making them go
through the planned (configurable) paths function.


> Locating the native code for collections requires some extra work.
> 
> 
> Someone mentioned that the "one application-specific directory"
> approach is better because it is easier to uninstall, but this
> argument doesn't really hold, since there are various package
> management systems to take care of this. I use one of the very
> simplest, GNU Stow.

(Off-topic: It *is*, obviously, easier to install and uninstall...
The unix thing has always been a PITA, not only for PLT but for
anything else -- starting from the problem of a single executable
namespace per machine (which makes it a single namespace for the whole
world), and ending at the decision to put all shared libraries in
/usr/lib, and use symlinks to resolve problems.  Judging by the single
stow-specific prefix you're using below, it looks like this stow thing
uses a single directory to know which files it spreads out later -- so
it uses a single directory to solve the unix-multiple-directories
problem.)


> With most autotools-based software, all I have to do is:
> 
> ./configure --prefix=$HOME/local
> make
> make install prefix=$HOME/local/stow/this-app
> cd $HOME/local/stow
> stow this-app
> 
> and everything just works.
> 
> However, with PLT it's not quite this simple. I do (approximately):
> 
> ./configure --prefix=$HOME/local
> make
> make install prefix=$HOME/local/stow/plt-lib
> 
> cd $HOME/local/stow/
> mkdir -p plt/lib
> mv plt-lib plt/lib/plt
> cd plt
> ln -s lib/include lib/man lib/bin .
> cd lib
> ln -s plt/lib/* .
> cd ../..
> stow plt
> 
> And then I probably run setup-plt at some point. This works
> reasonably well for my purposes: I get mzscheme into my path
> ($HOME/local/bin) which is usually all that matters to me. Still,
> it's a bit awkward.
> 
> So, before we start talking about "fixing" the file layout, we need
> to know what it should be changed to. I'd be rather surprised if
> everyone agreed that the layout I suggested would be an
> improvement...

Of course nobody would agree with you if they come from the
single-directory world of Windows or OSX, or if they're used to the
convenience of user-local installations by putting the plt directory
somewhere in your home dir and linking executables in your ~/bin/.
For (unix) machine installations, which include distro-specific
packaging like .debs and .rpms, I'd be surprised if anyone disagrees
(except on minor points).


And finally, this is the text I have.

-------------------------------------------------------------------------------
1. Make ./configure with a --prefix use an absolute path in the binaries ->
   also the resulting mzscheme should create absolute launchers

2. Make it possible to rewrite the binaries from an installer (so the sh
   installer can install either a single directory or absolute)

3. Maybe make it possible to tweak (`plt-' prefix) some names when installing
   in an asbolute path on Linux -- problematic names right now: `help-desk',
   `slideshow', `web-server*'.  `help-desk' is also problematic with its man
   page (might be simpler to just remove it).
   Lauri: Alternatively, make some binaries be marked as `helpers', which go to
          a different place.

4. Should change:
   ----> Installation will go to (eg) /usr:
   ---->   /usr/collects/... --> /usr/lib/plt/collects/...
                            *or* /usr/lib/plt/...
   ---->   /usr/bin/...      --> OK (but see #3)
   ---->   /usr/lib/...      --> /usr/lib/plt/... (note: collects here too)
   ---->   /usr/include/...  --> /usr/include/plt/...
   ---->   /usr/man/...      --> /usr/share/man/... or /usr/man/...
     ARI: autoconf currently sets manpages to go into /usr/man/ by default,
          Debian sets it to /usr/share/man with configure --mandir.  If you
          switch to a fully autoconf/automake system, those are all simple to
          change.
   ---->   /usr/notes/...    --> /usr/share/doc/plt/...
     ARI: /usr/share/doc is generally used for distribution-specific
          documentation, so applications don't usually install things there. I
          think it would be better to just put the documentation & notes in
          /usr/share/plt/{doc,notes} .
   * In each of the above, the `plt' part should be configurable, for example
     `pltVVV'.  (maybe the notes directory always has a version?)  Actually,
     the whole string should probably be configurable (what do standard
     configure scripts do?)
     ARI: Standard configure scripts don't generally change the
          application-specific directory name, but I'm sure it would be
          possible to add a --data-suffix parameter to configure to act along
          the same lines as the standard --program-suffix (which isn't often
          used)

   * Some things use the version as an extra directory level, like
     /usr/lib/plt/VVV/...
     ARI: This would probably be a better solution than /usr/lib/pltVVV.
          collects could go in /usr/lib/plt/VVV/collects/..., notes & docs
          could go into /usr/share/plt/VVV/..., and the rest could stay the
          same.  The core mred/mzgc/mzscheme libraries (probably not mzdyn.o)
          could even go into /usr/lib since they're versioned.

5. The previous prefix build mode should not exist -- it's the same as a normal
   build and copying/moving the result elsewhere.

6. The configuration/mapping function should have information on all of these,
   including notes and collects/doc, and including man for symmetry (and new
   installers?  Maybe make it #f if on Windows?)

7. Maybe make it possible to have (by defaults) something like
   /usr/lib/plt/site-plt/<more-collects-here> in addition to the main directory

8. Where should collects/doc go?

9. Maybe move in the general plt tree `collects/doc' to a toplevel `doc'

10. Get rid of the teachpack toplevel directory, -> make the add-teachpack menu
    entry go to collects/htdp or htdc by default (looks like htdch is too
    crowded to make sense for a teachpack dialog) -> move teachpack Docs to
    iplt/doc.
-------------------------------------------------------------------------------

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                  http://www.barzilay.org/                 Maze is Life!