<div dir="ltr"><div>Jay, </div><div>Thanks for the detailed summary. Right now I think the biggest missing piece of information when something goes wrong is "which packages were affected?". If I download package A, and it pulls in B and C, and they pull in D, E, and F, and then somewhere along the lines raco setup fails, I now have six packages installed, and no clear idea which ones are working fine and which ones need to be fixed. If raco pkg could communicate with raco setup in some way to associate files that failed with the packages they came from, it would be very useful at the end to get a report like: "raco pkg successfully installed packages B, E, and F; raco setup reports errors with the installation of A, C, and D." That or a "raco pkg status" command to look up similar information. Otherwise the user is left to track down the dependencies responsible for errors themselves, which can very time consuming when each step of the search involves running raco setup on some new subset of packages. </div><div>I'm sure this extra scrutiny and criticism for the package system isn't easy, so let me take this moment to also say -- thanks so much for writing the package system! It wouldn't be worth all this nitpicking if it weren't a great system; we've come a long way from distributing source code via ".plt" files, to having something worth using to bundle up our core software. </div><div class="gmail_extra"> <div>Carl Eastlund</div> <div class="gmail_quote">On Fri, Jun 14, 2013 at 10:13 AM, Jay McCarthy <<a href="mailto:jay.mccarthy@gmail.com" target="_blank">jay.mccarthy@gmail.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I'll try to respond to these four messages at the same time... Sam said, <div class="im">> In addition to the larger point Robby makes, this can be pretty > confusing. For example, you can fail to install enough dependencies, > I think. > > Another problem is that there's no way to know what to do to fix > things. Say there's an error in `raco setup` that's transient (the > machine lost power, for example). What command do I run to 'fix' the > setup? I don't even know what collections were being installed. </div>On Thu, Jun 13, 2013 at 9:27 PM, Robby Findler <div class="im"><<a href="mailto:robby@eecs.northwestern.edu">robby@eecs.northwestern.edu</a>> wrote: > I know it runs it. I don't know why Jay writes "The package system says > something is installed when the files are in place and the link is made. > From some perspective, that's its job.". I can't tell if there's some > technical piece I'm missing or not (on the surface, these words sound almost > lazy but I *KNOW* Jay is anything but lazy!) > > Robby > > > On Thu, Jun 13, 2013 at 10:21 PM, Carl Eastlund <<a href="mailto:cce@ccs.neu.edu">cce@ccs.neu.edu</a>> wrote: >> >> It does run 'raco setup', it just doesn't have much to do in response to a >> failure, at least right now. >> >> Carl Eastlund >> >> On Thu, Jun 13, 2013 at 11:14 PM, Robby Findler >> <<a href="mailto:robby@eecs.northwestern.edu">robby@eecs.northwestern.edu</a>> wrote: >>> >>> Also, Jay: can you explain more why 'raco setup' isn't something that we >>> should think about as running "inside" the pkg manager? (I'm not saying that >>> automatically rolling back packages is the right thing to do or anything >>> like that, but I would like to understand the model you have better.) </div>Short: I interpret these messages to be saying, "raco pkg printed error messages and didn't say what was wrong or what I should do about it". The messages came from 'raco setup', of course, and 'raco pkg' gave no interpretation. As a simple first step, we can have 'raco pkg' observe the output of 'raco setup' (which might require changing it) and if it gives any error message of any kind, end with a message like: "raco pkg detected strange output during setup. This could mean there was a problem. If you would like to undo this command, run: raco pkg remove the-package-you-installed" This sounds like a simple to change to make, but there's one big problem. After 'raco pkg update' runs, the undo command is much more complicated and scary in its own way raco pkg remove --force the-package-you-updated raco pkg install the-source-for-the-package-you-updated-at-the-old-git-commit But some package sources may not be possible to get back, such as an old tar ball that you downloaded and don't have any more. When raco pkg update runs normally, it leaves the old files in place until the end and then deletes them. It could put these into some sort of 'archive' location to facilitate that. But it's a big change and easy to get very wrong. Long: I also interpret these messages to be saying "When raco pkg returns with exit status 0, the packages I asked to be install/updated should 'work'." This is a reasonable wish, but I don't think it can be enforced technically. Let's un-ravel worked-ness: 1. Does the task I thought it would do Clearly no way to ensure this. No package system does this. 2. Every file executes without errors This is what DrDr normally does and is too burdensome for every package, and includes problems of the next stage. No package system does this. 3. Passes its test cases on my system CPAN is fairly unique in doing this upon install, but virtually no other package system does this. I don't think it is actually useful, because there's no reason to have confidence in the test cases and it's hard from a test case writer perspective to get everything in place. (For example, a Mongo DB interface needs a running Mongo DB server to test against.) 4. Compiles on my system Most package systems don't need to think about this, because they ship only binaries. Languages like Perl and Javascript also don't really do this, because they don't compile in any real sense. Languages that do can typically make a pretty simple promise based on their languages inability to have bizarre and unpredicatable compile-time dependencies. Racket has the ability to provide bizarre and unpredictable compile-time dependencies: #lang racket/base (define-syntax (macro-invocation stx) (if matthew-isn't-looking #'(require #,(random-file)) #'(void))) (macro-invocation) Since we can't really offer this, the best we can do is trust the dependencies listed. We have ideas on how to enforce those dependencies, but it isn't implemented yet. 5. Its actual dependencies are on my system This is like the above. 6. Its listed dependencies are on my system This is the standard that 'raco pkg' uses. It guarantees that either all dependencies are installed or that no changes were made. This means that you can never run 'raco pkg update', download a new version of an installed package that has a new dependency that conflicts with a current package, and then end up with a failure that results in some new packages and some old packages. This, by the way, is the most common standard of 'worked-ness' on Linux package systems, like pacman, apt-get, etc, that 'raco pkg' is based on. These end-user tools don't enforce anything when an install happens and sometimes there are mistakes by package developers that cause something to break on an end-user system and it has to be dealt with somewhere on the developer side. In other words, there is *no* error that can happen during 'raco setup' that breaks the package system. Sam's question about what do I run to "fix" things doesn't make sense to me, because there can't be anything wrong at that point. If you are using links and then go change the code in response to a 'raco setup' error, you don't need to tell the package system at all, you just run 'raco setup' (with no arguments so it runs on everything) again. -- Given that I think we can't enforce higher standards technically, my goal has been to enforce standard (2) with the concept of ring-0 packages. To remind you, ring-2: No constraints ring-1: Cannot conflict with any package in ring-1 or ring-0. ring-0: Must contain tests that I can run on DrDr and that pass reliably Once that is in place, the next step would to make it "difficult" to install anything other than ring-0 packages. (For instance, not offering auto-install and not having them in the central documentation site.) The goal would be to make it so that 'raco setup' never errors. <div class="HOEnZb"><div class="h5"> -- Jay McCarthy <<a href="mailto:jay@cs.byu.edu">jay@cs.byu.edu</a>> Assistant Professor / Brigham Young University <a href="http://faculty.cs.byu.edu/~jay" target="_blank">http://faculty.cs.byu.edu/~jay</a> "The glory of God is Intelligence" - D&C 93 </div></div></blockquote></div> </div></div>