[racket] improving speed of namespace attachment/requiring?

At Wed, 27 Feb 2013 16:05:25 -0700, Danny Yoo wrote:
> Hi everyone,
> 
> I'm working on making Whalesong be the evaluator for the WeScheme
> environment.  One problem I'm running into is the speed of REPL
> compilation.  I'm compiling on the server side, and I want the
> compiler to be stateless, so I create a clean namespace per individual
> interaction.
> 
> I'm doing something like this to create such namespaces:
> 
> ###################################
> #lang racket/base
> (define this-namespace (make-base-empty-namespace))
> (define (make-repl-namespace [module-path 'racket/base])
>   (parameterize ([current-namespace this-namespace])
>     (dynamic-require module-path #f))
>   (make-empty-namespace)
>   (define ns (make-empty-namespace))
>   (parameterize ([current-namespace ns])
>     (namespace-attach-module this-namespace module-path)
>     (namespace-require module-path))
>   ns)
> ###################################
> 
> 
> I create a single this-namespace to hold the instantiation of my base
> language module, and then inject it into a fresh namespace I construct
> per repl interaction.  That way, I instantiate my language module just
> once, rather than per interaction.  I note, though, that if I do this,
> the cost of attaching and requiring the module can still be a little
> expensive, depending on the language namespace I want to construct.
> 
> 
> For example, if I execute the following:
> 
> ###################################
> (for ([i 10])
>   (time (parameterize ([current-namespace (make-repl-namespace
> 'lang/htdp-beginner)])
>           (let ()
>             (compile
>              '(* x 3))))))
> ###################################
> 
> then I observe the following on the Racket console:
> 
> ######################
> cpu time: 256 real time: 275 gc time: 69
> cpu time: 22 real time: 23 gc time: 0
> cpu time: 63 real time: 65 gc time: 34
> cpu time: 21 real time: 22 gc time: 0
> cpu time: 20 real time: 20 gc time: 0
> cpu time: 38 real time: 41 gc time: 11
> cpu time: 23 real time: 23 gc time: 0
> cpu time: 43 real time: 44 gc time: 21
> cpu time: 24 real time: 26 gc time: 0
> cpu time: 17 real time: 17 gc time: 0
> ######################
> 
> This isn't so bad, on my laptop, but I expect the times to double at
> least when I deploy to my slower Amazon EC2 servers.  Can I do better
> here? If I switch the language here to 'racket/base, then it's a lot
> faster: why?
> 
> 
> I still have some tricks I can play (like memoizing the compiler).
> But anything I can do to keep the compilation cost down would be
> greatly appreciated.  Thanks!

I think you want to generate new namespaces that share the module
registry, which means that they share the visit of the module indicated
by `module-path'.

Below is one way to do it, although probably there should be an easier
way.

----------------------------------------

#lang racket/base

(define make-fresh-namespace (eval 
                              '(lambda () 
                                 (variable-reference->empty-namespace
                                  (#%variable-reference)))
                              (make-base-namespace)))

(define (make-repl-namespace [module-path 'racket/base])
  (define ns (make-fresh-namespace))
  (parameterize ([current-namespace ns])
    (namespace-require module-path))
  ns)

(for ([i 10])
  (time (parameterize ([current-namespace 
                        (make-repl-namespace 'lang/htdp-beginner)])
          (let ()
            (compile
             '(* x 3))))))