[racket-dev] Packaging

On Sat, Feb 19, 2011 at 9:39 AM, Jay McCarthy <jay.mccarthy at gmail.com> wrote:
> I've batch my responses to yesterday's questions together.
>
> As a general note, I'd like to have my document be an accurate
> reflection of what I should do when I start coding, so if you think I
> should update it to clarify the answers to these questions, please let
> me know. I'm blinded a bit by my intentions when I read it.
>
> 2011/2/18 Robby Findler <robby at eecs.northwestern.edu>:
>> Minor comment: why encourage names like "libgtk" and "libgtk2" instead
>> of a major and minor version number (ala PLaneT)? Don't we want those
>> two libraries to be associated somehow (at least loosely)?
>
> I think having libgtk and libgtk2 alleviates a confusion with the
> "most recent version" policy. For example, I write my package when
> there is only libgtk and I don't specify which version I want, because
> any will do. Later libgtk gets upgraded and its second major version
> is not fully backwards compatible. I don't think it is plausible for
> the reader of my package metadata to know that I didn't know about the
> new version and SHOULD have written a "major version 1 requirement",
> but didn't.
>
> By having a strict separation between major versions, I think it is
> kinder to less specific package metadata. (If you look around PLaneT,
> very few packages actually specify their dependencies with much
> detail.)
>
> As a slight social engineering, this policy also makes it easier to
> provide alternatives to dead packages, because upgrading to an
> alternative vendor is no harder than upgrading to the official
> version.

It looks to me like you there is relevant, important metadata that
you're making someone fold into an implicit place instead of an
explicit one.

Will you have a convention for these? What if I decide to call mine
"libgtk2.0" and someone else calls theirs "somepackage-2"? That
doesn't seem good fo

> Also, it may discourage new major versions and therefore
> encourage more compatibility.

This doesn't seem to follow. :)

>> Also, it also isn't clear which of the complaints with PLaneT you're
>> actually dealing with. I don't see anything about security for example
>> or the discovery issue.
>
> By moving package dependencies outside of modules and allowing a
> system-wide installation, I attack the second-class-nes of PLaneT. By
> a different default policy, I attack the upgrade path. By having a
> more exposed directory structure and looser package hosting
> constraints, I attack the improvement problem. On the social site by
> having reputation, reviews, and tags, I hope to solve the discovery
> problem. By allowing any URL (without a custom server) as a package
> source, I hope to solve the centralization problem. By making the
> packaging system more capable, I hope to decrease the size of the
> core. Once that happens, those packages can be independently
> upgraded. On the security defaults, by using URLs, we can use HTTPS
> to get slightly better security. I mention having checksums on package
> files to ensure the right file was got. By allowing private
> repositories, you can ensure that you control your security destiny. I
> solve the unintended installation problem by making package
> installation separate from the runtime, where we could provide an
> "apt-get" style of "these are the other packages that will be
> installed". I solve the module privacy with the linking directories.
>
> I have no solution for expensive installations (indeed in my final
> open problems I mention how we would store compiled code at the
> server.) The other part of installation---documentation---could be
> solved the same way we do now (explicit option to not build) and have
> better searching on the package site.
>
> My solution for untrustworth packages is a rather weak "black balling".
> I think sandboxing compilation is outside the short-term scope of this project.

These seem reasonable, but I did not get a clear connection when
reading your essay. My comment is probably a presentation one, then.
In general, the essay seems long on problem descriptions and short on
solution descriptions, relying on the reader to be familiar with
low-level details of Racket and having to be able to synthesize from
them how things will interact. I suspect that a high-level description
of the solution you're imagining (and possibly some more specific uses
cases) will make your essay more readable (as you may have noticed,
most of the replies don't actually say anything about your proposed
solution in any detail; probably because we can't understand it).

>> And one fairly abstract worry: can I end up in a situation where I had
>> a working system with dependencies between packages being "obvious"
>> but install a new package and fall out of the "obvious" realm in some
>> potentially confusing way?
>
> I think this is the most confusing situation:
>
> - Install package A that requires B
> - Develop a packageless program that uses B
> - Upgrade to a new provider of B's features... B2
>
> Your packageless program uses B2, but A still uses B, even though
> you may think of them as the same package. I think the alternative
> behavior, that A starts using a different provider, is worse though.

This kind of unintentional (possibly unwanted) upgrades seems
troublesome, esp when coupled with long dependency chains and the
sharing DAG's that's built into the Racket module system. It would be
nice if there were something we could do better. I don't have a
concrete idea (that isn't planet, which Sam's message reasonably
argues against) but it seems like we should be able to do better.
Perhaps things will be more clear when we have more experience with
the system, tho.

Robby