[plt-scheme] PlaneT, plts and distributing software.

Jacob Matthews wrote:
> There are two  safeguards in place to protect you in this situation. The 
> first is the linkage table: the first time a source file requires a 
> particular planet module with a particular version, that exact version 
> gets "linked" to that source file and it will always get exact package 
> from then on, even if a new, better, compatible package becomes 
> available. The user of the software has to explicitly say that they want 
> your package to get relinked to newer packages if they want that 
> behavior. 

Does this only apply to an existing installation, or is it possible to 
include the linkage table when you distribute a program in source form? 
  (That might be useful...)

> Second, if for whatever reason you want to use a planet 
> package but don't trust its maintainer to actually make allegedly 
> backwards-compatible releases truly backwards-compatible, you can 
> restrict your require-spec:
> 
>  (require (planet "file" ("usr" "pkg" 1 (= 3))))
> 
> means "exactly version 1.3, accept no substitutes." Similarly
> 
>  (require (planet "file" ("usr" "pkg" 1 (- 3))))
> 
> means "anything between 1.0 and 1.3" and
> 
>  (require (planet "file" ("usr" "pkg" 1 (4 8))))
> 
> means "anything between 1.4 and 1.8, but no higher or lower."

I'm confused about what the version numbers mean, now.  If I specify 
"exactly version 1.3", and the maintainer comes up with an upgrade which 
is supposed to be fully compatible with version 1.3, how does PLaneT 
distinguish between those two versions, and prevent me from getting the 
newer one even though I've specified (= 3)?  Assume a new install 
without the linkage table, if that makes sense.

> [These options don't seem to be widely known, but they've been in PLaneT 
> since the very beginning.]

I've only read the Help Desk (are there other docs/papers I should 
read?), and that info seems to mainly be contained in the grammar for 
PLaneT requires, but the meaning of version numbers doesn't seem to be 
explained.  Not a complaint, but that might explain the lack of 
widespread knowledge about this. ;)

>> I can see the benefit of trusting in PLaneT and, *in some cases*, 
>> requesting "whatever you guys think is compatible enough with X & Y". 
>> But that should be an option, not the only possibility.
> 
> 
> I think that ought to be the default, and there should be ways to 
> override the default ... that's why I designed the system the way I did. 

That's fine with me, assuming there are ways to override it reliably.

> On the other hand, I think you're right that the client ought to be able 
> to have some say in what planet package gets used to satisfy what 
> dependency also (or more generally, anyone who doesn't want to edit the 
> source code), and at the moment there's no good story for that.

OK, thanks.

> If you want it to be centralized *by default*, that's got to be 
> approached very carefully. One of the most important design principles 
> of PLaneT is that it needs to be as simple as it can possibly be. I  
> consider it critically important that you can copy and paste a single 
> line from planet.plt-scheme.org into your program-- without even 
> understanding what it means -- and have the system do the right thing. 
> As time goes by, you shouldn't have to realize that the web page's 
> require lines are all wrong and that if you really want to use planet 
> the _right_ way you should've been doing something else entirely all 
> along. That's not to say it's not a real problem, just that any solution 
> has got to be very careful not to give PLaneT a higher barrier to entry.

With the approach that Noel and I have been getting at, the paste from 
PLaneT could simply be a (require (lib ...)) statement, with the details 
of whether & when to look on PLaneT being specified somewhere else. 
That shouldn't later become the wrong approach, unless you have some 
specific need to embed the version numbers in source code, which I'm 
arguing should not be the default case.

I don't think that approach would have a higher barrier to entry, 
assuming PLT ships with a default PLaneT configuration that will allow 
users to just "require and go".

I do acknowledge that it adds complexity at some level to insert a layer 
of indirection, and making that transparent to users will require more 
work.  It's just that I tend to think that anything short of that is 
going to be papering over some problems.  Proper management of version 
dependencies in software is notoriously messy and subtle, and I think 
shortcuts are likely to come back to bite you (or worse, bite me! ;)

> ;; ----
> 
> So at this point there are at least a couple concrete major feature 
> requests outstanding for PLaneT, and a few more vague major requests:
> 
> Concrete:
> 1. Allow developers to store planet code in any directory they'd like 
> and tell planet how to get to it, so they can test their modules as they 
> develop them
> 2. Rearrange the server code to allow for mirrors, and then change the 
> client so that it knows about them
> 3. There are certain situations in which planet-managed .zo files can 
> screw up an upgrade or prevent you from working with two different 
> versions of PLT Scheme at once until you run setup-plt -- this is a bug 
> and needs to be fixed.
> 
> More general:
> 1. Give developers and users a better way to control what require spec 
> goes to what actual .plt file, when it gets downloaded, etc
> 2. Give developers some way to crystallize an entire project, 
> potentially consisting of multiple planet packages, into a single file 
> that can be distributed (i.e., compile your package using 
> statically-linked rather than dynamically-linked libraries, as it were)
> 
> Is that an accurate list of all the functionality people want out of 
> planet that isn't there yet? 

I think that does it for me.

> I ask because I want to start thinking 
> about how to make planet serve people's needs better, so I'd like to 
> hear more about what people don't have and want. I particularly want to 
> hear any actual problems people are constantly getting frustrated by.

I've been avoiding PLaneT because of having run into some of these 
issues previously.  My perspective was definitely influenced by having 
had to go through a number of modules and change PLaneT requires to 
standard requires, and/or vice versa.

One example of where I had to do this was in using SSAX with a program I 
had written a year or so ago, which depended on an older SSAX.  The PLT 
version of SSAX wasn't compatible, and fixing that was out of scope for 
what I was working on, so I tried the PLaneT version of SSAX, which 
involved changing all the requires.  This version appeared to work for a 
while during development, until I discovered another compatibility issue 
with my old code.  I ended up using an older .plt file, which meant that 
I had to change all the requires back again.

This was a perfect case where you might think embedding the version 
dependency in the source code solves the problem (assuming the required 
version were available via that approach).  However, if I know that the 
application as a whole depends on a certain version of SSAX, I don't 
want to encode that fact in multiple modules, because I'm likely to 
upgrade at some point in the future.  So what I want is to be able to 
say (require (lib ... "ssax")), and have that mean, for a particular 
application, a particular version of ssax.

The wrapper module approach could work, but I don't think it easily & 
reliably satisfies my desire to use the logical name "ssax" for the 
collection, without having to hardcode some qualifier which needs to be 
changed later.  This might sound picky, but I'm just telling you what I 
would consider ideal, as a developer.  I want my modules to use ssax, 
because I anticipate that one way or another, they'll "always" use ssax, 
and if I instead have to use my-ssax-wrapper or (ssax 0 9), I find that 
somewhat rebarbative.  (I've been waiting for a chance to use that word.)

Related to this, I also had an issue with ambiguity with PLT's default 
ssax module, which again had me wishing for an easy and reliable way to 
map logical collection names onto the filesystem.  Relying on precedence 
in collection paths is a bit fragile.  I suppose I could set up a 
per-application collection directory with symlinks to all the 
collections I want...

BTW, it may not surprise you to discover that for the same sorts of 
reasons, I don't even particularly like the idea of disambiguating by 
specifying, in a module, some identifying qualifier, such as in this 
case, "lisovsky" vs. say "plt".  Again, that choice to me is more of a 
"link-time" decision, and not something I want to repeat in multiple 
source modules.

That's my 2 cents, inflation-adjusted, with interest.

Anton