[racket-dev] experiment of the unified require syntax ==> Re: haskell's 'hell of a lot of libraries', planet

From: YC (yinso.chen at gmail.com)
Date: Mon Aug 9 18:12:42 EDT 2010

Hi all -

I have uploaded bzlib/UNIFY.plt to planet, hoping to demonstrate what an
unified require syntax between collects and planet *can* look like when we
keep it simple.  Please feel free to play with the code.

The example below represents how the syntax can look like (bolded are
translated to the planet form):

#lang planet bzlib/unify ;; hoping to see #lang racket here soon instead...

(require net/url ;; in collects
         *soegaard/infix* ;; from planet ==> (planet soegaard/infix)
         (only-in *bzlib/file* mtime) ;; ==> (only-in (planet bzlib/file)
mtime)
         )

(provide (all-from-out net/url ;; in collects
                       *soegaard/infix* ;; in planet
                       )
         )


Note the main difference is simply the removal of (planet).

bzlib/unify follows a very simple lookup rule (even simpler than I have
thought needed originally) - it checks to see if the module-path exists
(i.e. net/url or soegaard/infix) via the default
(current-module-name-resolver).  If it exists it's in collects, otherwise
wrap the path in `(planet ,module-path).  No additional database lookup is
required.  And it does not incur anymore network overhead for planet
packages than would have been needed in the first place (just the first-time
install).

The package is meant to showcase the viability of the approach and hope to
spark further conversations/actions.  Although the concept is doable as a
planet module, it should really be incorporated into the core module
resolver, since that can make it faster and do error handlings (hiding the
planet origin in an error message) better.

A few considerations:

1 - there might be name clashes between the packages in collects vs in
planet.

This can be fixed via some or all of the following approach:

   - reserve account names (such as racket or scheme) in planet so others
   cannot register them that maps to the collects packages so there are no
   future conflicts
   - if there are already conflicts - determine what to do there on case by
   case basis, including possibly taking over the account or rename the collect
   package in question

2 - planet packages are always in the form of <account>/<package>.plt,
whereas collects do not follow this rule.

This can either be fixed by changing the planet system to enable top level
packages, or change the collects package to follow the same convention.

3 - you (racket core devs) might want a new way to specify module paths
instead.

I am only focused on removing planet from the require/provide syntax, and
trying to show that it can be done without a hefty re-architecture so you
might decide to incorporate it.  If you do feel new module path formats are
warranted, that's great too.

Thoughts/comments/questions?  Love to discuss further on this topic.

Cheers,
yc

On Wed, Jul 28, 2010 at 12:33 PM, YC <yinso.chen at gmail.com> wrote:

>
> On Wed, Jul 28, 2010 at 1:09 AM, Stephen De Gabrielle <
> stephen.degabrielle at acm.org> wrote:
>
>
>> It's not to early to think about an expanded set of categories
>>
>
> One idea is to allow module writers to add to the categories or tags so it
> becomes a decentralized process, like how blogs do it these days.
>
> On Wed, Jul 28, 2010 at 6:41 AM, Matthias Felleisen <matthias at ccs.neu.edu>
>  wrote:
>
>
>> On Jul 28, 2010, at 12:26 AM, YC wrote:
>>
>> > Other package systems separate the installation step from the import
>> step
>>
>> Indeed, this is the key design decision separating us from the rest of the
>> world, and it is not clear whether it was a good decision.
>
>
> I personally like the design except for the issues I've raised so far, but
> the disadvantage can be remedied by unifying the require spec syntax between
> collects and planet.
>
> On Wed, Jul 28, 2010 at 7:03 AM, Dave Gurnell <d.j.gurnell at gmail.com>
>  wrote:
>
>
>> I'd also like to be able to set up a PLaneT proxy to add in-house
>> libraries in on the top of publicly available libraries from
>> planet.racket-lang.com.
>>
>
> See my bzlib/planet package.  This is an example where I would like to see
> the core team take advantage of what's available on planet directly instead
> of porting it into core.
>
> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van Dyke <neil at neilvandyke.org>
>  wrote:
>
>>
>> Most important for me, I'd like to be able to define multiple (what I'll
>> call for now) repositories (like Debian "apt").
>
>
> FWIW - bzlib/planet is designed for you to facilitate having your own
> internal packages now.  When multi-repo becomes  available, you can still
> use bzlib/planet to host your own repo.
>
> On Wed, Jul 28, 2010 at 7:03 AM, Dave Gurnell <d.j.gurnell at gmail.com>
>  wrote:
>
>
>> I'd personally be in favour of a leaner core distribution with more code
>> in external packages, so I can choose what I download when I'm only
>> interested in a single web application.
>>
>
> Once the collects and planet require syntax are unified - moving the
> currently-core packages to planet will become a snap. At that time it might
> even be possible to have "build your own distribution" by picking your own
> packages.
>
> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van Dyke <neil at neilvandyke.org>
>  wrote:
>
>
>> Then, somehow I would like to prevent packages in the contributed
>> repository from overriding those in the core and internal repositories.
>>  Guaranteeing this through naming, like Java packages, is one way, though
>> that could be cumbersome.
>>
>
> Now I think about it, it might make sense to have local repo able to
> overwrite core repo in case you want to "patch" the core code.  In that case
> the module resolve I proposed earlier would have to be changed to:
>
>    1. Try lookup the package in planet cache
>    2. If #1 fails, lookup in COLLECTS
>    3. If #2 fails, try lookup the package in planet repo
>
> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van Dyke <neil at neilvandyke.org>
>  wrote:
>
>
>> Package signing to authenticate the packager still seems useful, like it
>> did in the beginning, but that could be revisited in context of whatever
>> other improvements are made.
>>
>
> I like what ASDF has to offer here.  The only thing I would add is that
> this auth mechanism should be optional (perhaps as a policy per repo).
>
> All my additional 2 cents.  Cheers,
> yc
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/dev/archive/attachments/20100809/3e358313/attachment.html>

Posted on the dev mailing list.