[racket] Should we _always_ use `only-in` or `prefix-in` when `require`-ing packages?

I'll elaborate on Jay's answer --- because I had written most of this
by the time I saw Jay's post, and I think this version of the answer
adds a little.

Despite various efforts, we have no technical solution here, so far:

 * Always using `only-in` or `prefix-in` seems too painful. (We know
   that in some languages it's conventional to always prefix imports,
   but those languages don't consider things like `lambda` or the
   application form to be imports.)

 * When building up libraries like `racket/string`, `racket/list`, and
   especially `racket/base`, we generally try to avoid adding new
   exports, but each attempt to follow a strict "don't add new exports"
   rule has failed.

For now, we are trying out a social solution, instead of a technical
one:

 * Allow new exports as "backward-compatible", even though new exports
   can break old programs.

 * Rely on package producers to make sensible choices about when new
   exports are necessary.

 * Rely on package consumers to update as necessary to fix the
   occasional collision.

It's not obvious that this will work. Nevertheless, our current
thinking is that this social approach might be better overall than the
technical approaches we have at hand. Meanwhile, experiments continue
toward a technical solution.

The Racket developers have been following this approach for years. So,
to put it all another way: We sometimes add a new binding to
`racket/base', because putting that binding somewhere else would be too
painful. The `racket/base` library is not supposed to be special, which
means that everyone gets to add new bindings to a library when they
think it's necessary.

Probably, the approach works as well as it has because we add exports
sparingly, it's easy to fix problems created by collisions (compared to
many other kinds of incompatibilities), and a library that is
maintained enough to be useful is easily maintained with such fixes.


At Mon, 18 Nov 2013 08:45:59 -0700, Jay McCarthy wrote:
> I explicitly mentioned this in the RacketCon talk. PLT generally does
> not consider new exports to be backwards incompatible, even though in
> principle it is.
> 
> I think the only-in/prefix-in idea is the right way to go, but it is
> pretty painful. There are a few ways to make it simpler. I have a
> "interface" experiment that makes it so you can define an interface to
> a module separate from its implementation:
> 
> https://github.com/jeapostrophe/exp/blob/master/interface/interface-in.rkt
> 
> I think that using such a system in a more integrated way throughout
> Racket is part of "Racket 2".
> 
> Jay
> 
> 
> 
> 
> 
> On Mon, Nov 18, 2013 at 8:22 AM, Greg Hendershott
> <greghendershott at gmail.com> wrote:
> > Pre-caveats:
> > - Maybe this has been discussed before?
> > - This is true (IIUC) of the old Planet as well as the new package system.
> > - Maybe it's true of every package system.
> >
> > With that said:
> >
> > Technically, to maintain backward compatibility, a package should
> > _never_ `provide` a new name in an existing file. Either that or,
> > package users should _always_ use the `only-in` or `prefix-in` forms
> > of `require`.
> >
> > Why?
> >
> > 1. My package defines `foo`. I `(require other-package)` (i.e.
> > other-package/main.rkt) which today does not provide any `foo`.  I
> > list other-package as a `dep` in info.rkt.
> >
> > 2. Someday, other-package add and starts to provide a `foo`, too. (I
> > don't notice. How/when/why would I?
> >
> > 3. Soon after, a user installs my package for the first time, and in
> > the process the dependency other-package is automatically installed --
> > latest version. Result: It breaks, `foo` is already defined.
> >
> > If instead my package had done `(require (only-in other-package things
> > ...))`, everything would have been fine: I wouldn't get the other
> > `foo`. Or, if I'd used `(require (prefix-in x: other-package))`, I
> > would harmlessly get the other `foo` as `x:foo`.
> >
> >
> > Is this correct?  Because it seems like something that needs to be in
> > bold type in the package docs: Simple Practical Advice.
> >
> > (Otherwise, for many people, I think the notion of backward
> > compatibility is "don't remove or change old stuff".  Not, "don't add
> > new stuff".  So it might not be obvious to everyone. It wasn't to me,
> > until thinking about this recently.)
> >
> >
> > p.s. Even in the case where I _know_ that `foo` will soon be provided
> > in other-package -- let's say I submit a PR for my handy `foo` to be
> > added to that package, with a view toward eventually dropping my own
> > version -- the timing could be tricky. There's a window where a given
> > user might get the wrong combo of package versions. Using `only-in` or
> > `prefix-in` allows there to be a bridge period where it can exist in
> > both packages.
> >
> > p.p.s. I suppose another strategy is a rule for pkg devs: Never
> > `provide` new things in an existing file. Always create a new file,
> > and put the `provide` there. To get it, users must `require` the new
> > file.  This is a sort of "`only-in` by others means".  However it also
> > means a convenient all-in-on main.rkt is no longer an option. This
> > rule seems to mean a proliferation of files, because once you deploy
> > the package, the new-file rule kicks in again. As a result, it seems
> > like the only practical solution is in the hands of package users.
> > ____________________
> >   Racket Users list:
> >   http://lists.racket-lang.org/users
> 
> 
> 
> -- 
> Jay McCarthy <jay at cs.byu.edu>
> Assistant Professor / Brigham Young University
> http://faculty.cs.byu.edu/~jay
> 
> "The glory of God is Intelligence" - D&C 93
> ____________________
>   Racket Users list:
>   http://lists.racket-lang.org/users