[racket-dev] PLTCOMPILEDROOTS

One option may be to use the new "demodularizer" that my student Blake
Johnson is working on.

Basically, you give it the root of your application (main.rkt) and it
creates a new zo (compiled/main_rkt.zo) that has no dependencies but
"copies" [*] all the code in that would otherwise be required once the
program starts.

I put the [*] next to copy because it doesn't copy code from phases
you don't require and it does a light form of dead-code elimination to
remove even more. We've found that for a simple Web app, normally
about 3MBs of zos get loaded, but after demodularization only about
25k of zos get loaded. We're currently working on getting the Racket
optimizer to recompile this smaller code base to get more performance
out of it to, but we're not done yet. [Not done yet = of about 7000
Racket files, 6 of them use features we haven't gotten working yet.]

If your Racket code wasn't changing a lot, then you'd be able to use
this to limit the number of zos necessary to just the one, plus
whatever "racket" itself needs to start. If your code changes a lot,
but your dependency on Racket doesn't often, we could work on a
modified version that would cache all your Racket-collect dependency
in to one giant zo that was required for all the Racket deps you have.

Jay

On Tue, Oct 9, 2012 at 1:22 PM, Dan Liebgold <dan.liebgold at gmail.com> wrote:
> On Tue, Oct 9, 2012 at 11:32 AM, Matthew Flatt <mflatt at cs.utah.edu> wrote:
>>
>> At Tue, 9 Oct 2012 18:00:14 +0000 (UTC), Dan Liebgold wrote:
>> It still seems strange to me that we should implement our own
>> filesystem of some sort to get decent filesystem performance.
>>
>
> Indeed. I think in game development we tend to be a bit nuts about
> performance, but I do have some numbers to back it up!
>
> During a single startup of our app we trigger about 7000 file operations in
> the collects directory. When stored on a local hard drive it takes about 2
> seconds. On a network drive that number climbs to about 30 seconds. And that
> is just loading the collects we utilize.
>
> We invoke Racket about 1500 times every time we build one of our game
> projects, and currently we have about 4-6 automated builds plus 20
> programmers building as well. (You can probably only imagine how much we
> depend on precompiled headers on the C++ side of our projects.) The numbers
> climb quickly. So accessing over the network is too slow.
>
> Now it turns into a bit of a distribution problem. We have 150+ people
> in-house that need to run this, plus 50 or so more outsourced in various
> locations around the country. The standard Racket distro contains about
> 13,000 files. That many copies and syncs gets quite awkward, although it
> works.
>
> Ideally we would have a handful of executables and archives to distribute
> and utilize. Also ideally, the archives would contain just .zo files (or
> something equivalent.)
>
> Now I realize that we are the ones who must lay the track in front of this
> speeding freight train, so we are willing to do what it takes to implement
> this... just point me in the right direction!
>
> BTW, when I selected MzScheme for this project back in '05 I had no idea
> both what our production would look like at this point nor how well MzScheme
> would operate under these constraints. And Racket is improving upon it in
> every way!
>
> --
> Dan Liebgold    [dan.liebgold at gmail.com]
>
> _________________________
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
>



-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

"The glory of God is Intelligence" - D&C 93