[racket-dev] experiment of the unified require syntax ==> Re: haskell's 'hell of a lot of libraries', planet

From: Jay McCarthy (jay.mccarthy at gmail.com)
Date: Thu Aug 12 17:17:20 EDT 2010

Hi YC,

At the meeting we had in Chicago this past weekend, we discussed the
packaging system and where it needs to be. One of the high-level
points was that there should be no distinction between core and user
packages in a program; the only distinction between these will be in
the installation environment. I think that's one of your main desires,
so I hope that will make you happy.

In the next few months, I plan on sending out the design document and
specification of the packaging system for comments and I hope that you
will read it closely.


On Mon, Aug 9, 2010 at 4:12 PM, YC <yinso.chen at gmail.com> wrote:
> Hi all -
> I have uploaded bzlib/UNIFY.plt to planet, hoping to demonstrate what an
> unified require syntax between collects and planet can look like when we
> keep it simple.  Please feel free to play with the code.
> The example below represents how the syntax can look like (bolded are
> translated to the planet form):
> #lang planet bzlib/unify ;; hoping to see #lang racket here soon instead...
> (require net/url ;; in collects
>          soegaard/infix ;; from planet ==> (planet soegaard/infix)
>          (only-in bzlib/file mtime) ;; ==> (only-in (planet bzlib/file)
> mtime)
>          )
> (provide (all-from-out net/url ;; in collects
>                        soegaard/infix ;; in planet
>                        )
>          )
> Note the main difference is simply the removal of (planet).
> bzlib/unify follows a very simple lookup rule (even simpler than I have
> thought needed originally) - it checks to see if the module-path exists
> (i.e. net/url or soegaard/infix) via the default
> (current-module-name-resolver).  If it exists it's in collects, otherwise
> wrap the path in `(planet ,module-path).  No additional database lookup is
> required.  And it does not incur anymore network overhead for planet
> packages than would have been needed in the first place (just the first-time
> install).
> The package is meant to showcase the viability of the approach and hope to
> spark further conversations/actions.  Although the concept is doable as a
> planet module, it should really be incorporated into the core module
> resolver, since that can make it faster and do error handlings (hiding the
> planet origin in an error message) better.
> A few considerations:
> 1 - there might be name clashes between the packages in collects vs in
> planet.
> This can be fixed via some or all of the following approach:
> reserve account names (such as racket or scheme) in planet so others cannot
> register them that maps to the collects packages so there are no future
> conflicts
> if there are already conflicts - determine what to do there on case by case
> basis, including possibly taking over the account or rename the collect
> package in question
> 2 - planet packages are always in the form of <account>/<package>.plt,
> whereas collects do not follow this rule.
> This can either be fixed by changing the planet system to enable top level
> packages, or change the collects package to follow the same convention.
> 3 - you (racket core devs) might want a new way to specify module paths
> instead.
> I am only focused on removing planet from the require/provide syntax, and
> trying to show that it can be done without a hefty re-architecture so you
> might decide to incorporate it.  If you do feel new module path formats are
> warranted, that's great too.
> Thoughts/comments/questions?  Love to discuss further on this topic.
> Cheers,
> yc
> On Wed, Jul 28, 2010 at 12:33 PM, YC <yinso.chen at gmail.com> wrote:
>> On Wed, Jul 28, 2010 at 1:09 AM, Stephen De
>> Gabrielle <stephen.degabrielle at acm.org> wrote:
>>> It's not to early to think about an expanded set of categories
>> One idea is to allow module writers to add to the categories or tags so it
>> becomes a decentralized process, like how blogs do it these days.
>> On Wed, Jul 28, 2010 at 6:41 AM, Matthias
>> Felleisen <matthias at ccs.neu.edu> wrote:
>>> On Jul 28, 2010, at 12:26 AM, YC wrote:
>>> > Other package systems separate the installation step from the import
>>> > step
>>> Indeed, this is the key design decision separating us from the rest of
>>> the world, and it is not clear whether it was a good decision.
>> I personally like the design except for the issues I've raised so far, but
>> the disadvantage can be remedied by unifying the require spec syntax between
>> collects and planet.
>> On Wed, Jul 28, 2010 at 7:03 AM, Dave
>> Gurnell <d.j.gurnell at gmail.com> wrote:
>>> I'd also like to be able to set up a PLaneT proxy to add in-house
>>> libraries in on the top of publicly available libraries
>>> from planet.racket-lang.com.
>> See my bzlib/planet package.  This is an example where I would like to see
>> the core team take advantage of what's available on planet directly instead
>> of porting it into core.
>> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van
>> Dyke <neil at neilvandyke.org> wrote:
>>> Most important for me, I'd like to be able to define multiple (what I'll
>>> call for now) repositories (like Debian "apt").
>> FWIW - bzlib/planet is designed for you to facilitate having your own
>> internal packages now.  When multi-repo becomes  available, you can still
>> use bzlib/planet to host your own repo.
>> On Wed, Jul 28, 2010 at 7:03 AM, Dave
>> Gurnell <d.j.gurnell at gmail.com> wrote:
>>> I'd personally be in favour of a leaner core distribution with more code
>>> in external packages, so I can choose what I download when I'm only
>>> interested in a single web application.
>> Once the collects and planet require syntax are unified - moving the
>> currently-core packages to planet will become a snap. At that time it might
>> even be possible to have "build your own distribution" by picking your own
>> packages.
>> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van
>> Dyke <neil at neilvandyke.org> wrote:
>>> Then, somehow I would like to prevent packages in the contributed
>>> repository from overriding those in the core and internal repositories.
>>>  Guaranteeing this through naming, like Java packages, is one way, though
>>> that could be cumbersome.
>> Now I think about it, it might make sense to have local repo able to
>> overwrite core repo in case you want to "patch" the core code.  In that case
>> the module resolve I proposed earlier would have to be changed to:
>> Try lookup the package in planet cache
>> If #1 fails, lookup in COLLECTS
>> If #2 fails, try lookup the package in planet repo
>> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van
>> Dyke <neil at neilvandyke.org> wrote:
>>> Package signing to authenticate the packager still seems useful, like it
>>> did in the beginning, but that could be revisited in context of whatever
>>> other improvements are made.
>> I like what ASDF has to offer here.  The only thing I would add is that
>> this auth mechanism should be optional (perhaps as a policy per repo).
>> All my additional 2 cents.  Cheers,
>> yc
> _________________________________________________
>  For list-related administrative tasks:
>  http://lists.racket-lang.org/listinfo/dev

Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University

"The glory of God is Intelligence" - D&C 93

Posted on the dev mailing list.