[racket-dev] experience using the `pkg` branch

On Thu, Jun 13, 2013 at 10:56 PM, Jay McCarthy <jay.mccarthy at gmail.com> wrote:
> On Thu, Jun 13, 2013 at 3:56 PM, Sam Tobin-Hochstadt <samth at ccs.neu.edu> wrote:
>> As part of my experiment in creating a different split of the
>> repository into packages, I spent some time working with the new setup
>> for building Racket, and cut myself on many of it's rough edges.  Some
>> of these are about the package system in general, and some are about
>> the new repository and build infrastructure.
>>
>> * When there's an installation error, it's really hard to tell what
>> state you're in, or what you should do to fix it.  Usually, the
>> package manager just says that it's already installed.
>>
>> This is a general issue in the package system, and it's easily the
>> most significant problem I encountered. The package system decides
>> that something is installed very early, and many errors are possible
>> after that. If the package didn't build, or it's dependencies didn't
>> build, or any of many other things happened, the package system thinks
>> everything is ok.
>
> The package system says something is installed when the files are in
> place and the link is made. From some perspective, that's its job.
> Knowing whether 'raco setup' will succeed after those are in place
> isn't really possible beforehand and I don't think the idea of 'run
> raco setup and uninstall if it fails' is robust against the sorts of
> realistic and innocuous problems that raco setup can report. Maybe you
> could be more specific about what error you saw and what you wish had
> happened.

In addition to the larger point Robby makes, this can be pretty
confusing.  For example, you can fail to install enough dependencies,
I think.

Another problem is that there's no way to know what to do to fix
things.  Say there's an error in `raco setup` that's transient (the
machine lost power, for example).  What command do I run to 'fix' the
setup? I don't even know what collections were being installed.

>> * The error message when you look for a missing collection is really
>> long if you have a lot of packages installed
>>
>> * Using parallel make, ie `make -j 8` doesn't work from the new
>> top-level Makefile.
>>
>> * Even though unix systems don't use them, the extra libraries are
>> downloaded and packaged up on unix platforms while building.
>>
>> * When installing a package with a long chain of dependencies, this
>> message gets printed a lot:
>>
>> The following packages are listed as dependencies, but are not
>> currently installed,
>> so they will be automatically installed:
>>     at-exp-lib syntax-color
>> The following packages are listed as dependencies, but are not
>> currently installed,
>> so they will be automatically installed:
>>     parser-tools
>>
>> and so on ...
>>
>> We should just collect all the packages to be installed and print them
>> all at once (and perhaps prompt, the way `apt-get` does).
>
> Since package dependencies are in the package and not otherwise
> available, the manager is asking for permission to do a download.
> After doing the download, it can then look at the dependencies and see
> if there more. 'apt-get', etc have more monolithic repository
> structures than our package system, so they can analyze everything
> locally and compute the transitive closure. Furthermore, 'raco pkg' is
> prompting you, but you have it set on "auto" (the default) so it's
> equivalent to 'yes | apt-get'

Ah, that makes a lot of sense.

Could we do something different in "auto" mode to just print out the
packages that were installed at the end?

>> * I think it's possible to confuse the local catalog by moving
>> directories around in the `pkg` directory.
>>
>> * `make local-catalog` is not incremental -- it always re-downloads
>> and re-packs the external libraries.
>>
>> * Sometimes, errors can be hard to trace to a particular package.  For
>> example, this one was because there was a mistake in the
>> future-visualizer package info.rkt file.  But I had to go groveling in
>> /var/tmp/ to figure that out.
>>
>> /var/tmp/pkg13711534991371153499937/info.rkt::212: read: expected a closing '"'
>>   context...:
>>    /home/samth/tmp/pkg-racket/racket/lib/collects/syntax/module-reader.rkt:176:17:
>> body
>>    /home/samth/tmp/pkg-racket/racket/lib/collects/syntax/module-reader.rkt:173:2:
>> wrap-internal
>>    lang:read
>>    /home/samth/tmp/pkg-racket/racket/lib/collects/setup/getinfo.rkt:50:8: temp32
>>    /home/samth/tmp/pkg-racket/racket/lib/collects/setup/getinfo.rkt:37:0:
>> get-info/full/ext
>>    /home/samth/tmp/pkg-racket/racket/lib/collects/setup/getinfo.rkt:33:0
>>    /home/samth/tmp/pkg-racket/racket/lib/collects/pkg/lib.rkt:1274:0:
>> pkg-single-collection85
>
> Are there other examples? This stack trace is useful to just put a
> with-handlers at the right spot.

I got plenty of `raco setup` errors that were hard to track down in
the same way. Those don't have a stacktrace, though.  I can't remember
any others that were like this one.

Sam