[racket-dev] experiment of the unified require syntax ==> Re: haskell's 'hell of a lot of libraries', planet

Hi YC,

At the meeting we had in Chicago this past weekend, we discussed the
packaging system and where it needs to be. One of the high-level
points was that there should be no distinction between core and user
packages in a program; the only distinction between these will be in
the installation environment. I think that's one of your main desires,
so I hope that will make you happy.

In the next few months, I plan on sending out the design document and
specification of the packaging system for comments and I hope that you
will read it closely.

Jay

On Mon, Aug 9, 2010 at 4:12 PM, YC <yinso.chen at gmail.com> wrote:
> Hi all -
> I have uploaded bzlib/UNIFY.plt to planet, hoping to demonstrate what an
> unified require syntax between collects and planet can look like when we
> keep it simple.  Please feel free to play with the code.
> The example below represents how the syntax can look like (bolded are
> translated to the planet form):
>
> #lang planet bzlib/unify ;; hoping to see #lang racket here soon instead...
> (require net/url ;; in collects
>          soegaard/infix ;; from planet ==> (planet soegaard/infix)
>          (only-in bzlib/file mtime) ;; ==> (only-in (planet bzlib/file)
> mtime)
>          )
> (provide (all-from-out net/url ;; in collects
>                        soegaard/infix ;; in planet
>                        )
>          )
>
> Note the main difference is simply the removal of (planet).
> bzlib/unify follows a very simple lookup rule (even simpler than I have
> thought needed originally) - it checks to see if the module-path exists
> (i.e. net/url or soegaard/infix) via the default
> (current-module-name-resolver).  If it exists it's in collects, otherwise
> wrap the path in `(planet ,module-path).  No additional database lookup is
> required.  And it does not incur anymore network overhead for planet
> packages than would have been needed in the first place (just the first-time
> install).
> The package is meant to showcase the viability of the approach and hope to
> spark further conversations/actions.  Although the concept is doable as a
> planet module, it should really be incorporated into the core module
> resolver, since that can make it faster and do error handlings (hiding the
> planet origin in an error message) better.
> A few considerations:
> 1 - there might be name clashes between the packages in collects vs in
> planet.
> This can be fixed via some or all of the following approach:
>
> reserve account names (such as racket or scheme) in planet so others cannot
> register them that maps to the collects packages so there are no future
> conflicts
> if there are already conflicts - determine what to do there on case by case
> basis, including possibly taking over the account or rename the collect
> package in question
>
> 2 - planet packages are always in the form of <account>/<package>.plt,
> whereas collects do not follow this rule.
> This can either be fixed by changing the planet system to enable top level
> packages, or change the collects package to follow the same convention.
> 3 - you (racket core devs) might want a new way to specify module paths
> instead.
> I am only focused on removing planet from the require/provide syntax, and
> trying to show that it can be done without a hefty re-architecture so you
> might decide to incorporate it.  If you do feel new module path formats are
> warranted, that's great too.
> Thoughts/comments/questions?  Love to discuss further on this topic.
> Cheers,
> yc
> On Wed, Jul 28, 2010 at 12:33 PM, YC <yinso.chen at gmail.com> wrote:
>>
>> On Wed, Jul 28, 2010 at 1:09 AM, Stephen De
>> Gabrielle <stephen.degabrielle at acm.org> wrote:
>>>
>>> It's not to early to think about an expanded set of categories
>>
>> One idea is to allow module writers to add to the categories or tags so it
>> becomes a decentralized process, like how blogs do it these days.
>>
>> On Wed, Jul 28, 2010 at 6:41 AM, Matthias
>> Felleisen <matthias at ccs.neu.edu> wrote:
>>>
>>> On Jul 28, 2010, at 12:26 AM, YC wrote:
>>>
>>> > Other package systems separate the installation step from the import
>>> > step
>>>
>>> Indeed, this is the key design decision separating us from the rest of
>>> the world, and it is not clear whether it was a good decision.
>>
>> I personally like the design except for the issues I've raised so far, but
>> the disadvantage can be remedied by unifying the require spec syntax between
>> collects and planet.
>> On Wed, Jul 28, 2010 at 7:03 AM, Dave
>> Gurnell <d.j.gurnell at gmail.com> wrote:
>>>
>>> I'd also like to be able to set up a PLaneT proxy to add in-house
>>> libraries in on the top of publicly available libraries
>>> from planet.racket-lang.com.
>>
>> See my bzlib/planet package.  This is an example where I would like to see
>> the core team take advantage of what's available on planet directly instead
>> of porting it into core.
>>
>> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van
>> Dyke <neil at neilvandyke.org> wrote:
>>>
>>> Most important for me, I'd like to be able to define multiple (what I'll
>>> call for now) repositories (like Debian "apt").
>>
>> FWIW - bzlib/planet is designed for you to facilitate having your own
>> internal packages now.  When multi-repo becomes  available, you can still
>> use bzlib/planet to host your own repo.
>> On Wed, Jul 28, 2010 at 7:03 AM, Dave
>> Gurnell <d.j.gurnell at gmail.com> wrote:
>>>
>>> I'd personally be in favour of a leaner core distribution with more code
>>> in external packages, so I can choose what I download when I'm only
>>> interested in a single web application.
>>
>> Once the collects and planet require syntax are unified - moving the
>> currently-core packages to planet will become a snap. At that time it might
>> even be possible to have "build your own distribution" by picking your own
>> packages.
>> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van
>> Dyke <neil at neilvandyke.org> wrote:
>>>
>>> Then, somehow I would like to prevent packages in the contributed
>>> repository from overriding those in the core and internal repositories.
>>>  Guaranteeing this through naming, like Java packages, is one way, though
>>> that could be cumbersome.
>>
>> Now I think about it, it might make sense to have local repo able to
>> overwrite core repo in case you want to "patch" the core code.  In that case
>> the module resolve I proposed earlier would have to be changed to:
>>
>> Try lookup the package in planet cache
>> If #1 fails, lookup in COLLECTS
>> If #2 fails, try lookup the package in planet repo
>>
>> On Wed, Jul 28, 2010 at 8:20 AM, Neil Van
>> Dyke <neil at neilvandyke.org> wrote:
>>>
>>> Package signing to authenticate the packager still seems useful, like it
>>> did in the beginning, but that could be revisited in context of whatever
>>> other improvements are made.
>>
>> I like what ASDF has to offer here.  The only thing I would add is that
>> this auth mechanism should be optional (perhaps as a policy per repo).
>> All my additional 2 cents.  Cheers,
>> yc
>>
>
>
>
> _________________________________________________
>  For list-related administrative tasks:
>  http://lists.racket-lang.org/listinfo/dev
>



-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://teammccarthy.org/jay

"The glory of God is Intelligence" - D&C 93