[racket] Dynamic Templates?

To me it doesn't look like any specific imports are especially expensive,
but there are a bunch of them, so they add up.

But, preliminary testing suggests that this technique works well.

I imported all the invariant modules into the current namespace and stored
a reference to the current namespace, as you suggested:

(require mod1 mod2 ... modn)
(define original-ns (current-namespace))

Then, once inside the eval namespace, attached them all:

(map (λ(mod-name) (namespace-attach-module original-ns mod-name)) '(mod1
mod2 ... modn))

Now, instead of 7-8 seconds, each trip through eval takes 2-4 seconds.
Probably more can be done, but that's already much better.


Matthew Butterick



On Mon, Oct 14, 2013 at 4:24 PM, Robby Findler
<robby at eecs.northwestern.edu>wrote:

>
>
>
> On Mon, Oct 14, 2013 at 6:20 PM, Matthew Butterick <mb.list.acct at gmail.com
> > wrote:
>
>> I reviewed the docs about the module registry<http://docs.racket-lang.org/reference/syntax-model.html?q=module%20registry#%28tech._module._registry%29>,
>> but just so I understand the principle behind this technique --
>>
>> When you call (namespace-attach-module orig-ns 'module-name), what
>> you're saying to the new namespace is "if you need module-name, use the one
>> that's already been imported into orig-ns". Is this right?
>>
>>
> Right.
>
>
>> And if so, do I improve performance by adding another (namespace-attach-module
>> ...) declaration for each module that will never need to be reloaded in the
>> eval namespace?
>>
>>
> Right.
>
>
>> BTW I tried your suggestion on the quasicode, and it was indeed faster.
>> However, when I moved it back into the real code, there was no improvement
>> -- in fact, the real code became slower. Running the real code through
>> profile suggests that most of the time is being spent traversing the
>> imports (which is consistent with my observation that the complexity of the
>> page itself doesn't count for much). So whatever I can do to avoid
>> re-importing modules will probably be the winning technique.
>>
>>
> Which module's imports, do you know?
>
> Robby
>
>
>> Thanks.
>>
>>
>> On Mon, Oct 14, 2013 at 11:22 AM, Robby Findler <
>> robby at eecs.northwestern.edu> wrote:
>>
>>> Did you try using namespace-attach-module to attach 'racket and
>>> 'web-server/templates? Something like:
>>>
>>>   (define orig-ns (current-namespace))
>>>   (parameterize ([current-namespace (make-base-empty-namespace)])
>>>     (namespace-attach-module orig-ns 'racket)
>>>     (namespace-attach-module orig-ns 'web-server/templates)
>>>     (namespace-require 'racket)
>>>     (eval `(begin
>>>              (require web-server/templates)
>>>              (require ,my-source-path-string)
>>>              (include-template ,my-template-path-string))
>>>           (current-namespace)))
>>>
>>> But 7 seconds seems remarkably long (I see a speed up of about 50% for
>>> that function (with the my-source-path-string and my-template-path-string
>>> lines commented out): from 250 to 170 msec).
>>>
>>> It may be that your .zo files or something are not up to date. So you
>>> may want to stick a 'raco make' somewhere in there to avoid recompiling
>>> your files every time and just recompile the ones that changed.
>>>
>>> Robby
>>>
>>>
>>>
>>>
>>> On Mon, Oct 14, 2013 at 1:14 PM, Matthew Butterick <
>>> mb.list.acct at gmail.com> wrote:
>>>
>>>> At RacketCon I talked about my Pollen system for making web pages.
>>>> Though I don't use the Racket webserver to provide live generation of web
>>>> pages, I do use it to provide a kind of browser-assisted REPL for
>>>> development. As I edit the project files, I want to reload the URL in the
>>>> browser and see the changes (without restarting the webserver).
>>>>
>>>> Moreover, since it's a development tool, it wants to be maximally
>>>> dynamic. For instance, I might be editing source files containing page
>>>> content, or the design templates (which might also contain Racket code), or
>>>> supporting Racket libraries (e.g., CSS generators).
>>>>
>>>> Jay has suggested that this can be done with eval. And indeed it can.
>>>> Here's the quasicode I use:
>>>>
>>>>   (parameterize ([current-namespace (make-base-empty-namespace)])
>>>>     (namespace-require 'racket)
>>>>     (eval `(begin
>>>>              (require web-server/templates)
>>>>              (require ,my-source-path-string)
>>>>              (include-template ,my-template-path-string))
>>>>           (current-namespace))))
>>>>
>>>> This works great, with one wrinkle: it's the slowest operation in the
>>>> whole system, taking about 7-8 secs to render each page. This is not a
>>>> grave hardship, and certainly faster than restarting the web server. But
>>>> I'd still be curious if I'm overlooking a better approach.
>>>>
>>>> In general, the render time is fairly consistent regardless of what's
>>>> in the page, which suggests to me that most of the expense comes from
>>>> setting up the fresh namespace. I did try making a namespace separately and
>>>> reusing it, but that didn't work (code wouldn't get reloaded). Is there a
>>>> way to "clean" a namespace that's cheaper than setting up a new one?
>>>>
>>>> Another option I considered is starting up another instance of Racket
>>>> in a subprocess, but it would need to be made into a synchronous operation.
>>>> Long drive for a short day at the beach, etc.
>>>>
>>>> It's also possible that I'm running up against the irreducible cost of
>>>> high-quality eval, in which case, I will accept it and move on.
>>>>
>>>>
>>>> Matthew Butterick
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 6, 2013 at 7:36 PM, Greg Hendershott <
>>>> greghendershott at gmail.com> wrote:
>>>>
>>>>> Great timing! It's been on my short list to make my static blog
>>>>> generator more flexible. From
>>>>> http://docs.racket-lang.org/web-server/templates.html I got the idea
>>>>> that it would be suitable only for static templates, but now I
>>>>> understand that's not the case. Today I made a topic branch using this
>>>>> approach that's working well, which I plan to merge after some testing
>>>>> and polishing.
>>>>>
>>>>> p.s. Maybe it's obvious, but because `include-template` uses
>>>>> `include-at/relative-to/reader`, it won't include a template if you
>>>>> give it a full path. Maybe there's a more-elegant way, but I found it
>>>>> worked to parameterize `current-directory` around the call to `eval`.
>>>>>
>>>>>
>>>>> On Wed, Jun 5, 2013 at 4:45 PM, Jay McCarthy <jay.mccarthy at gmail.com>
>>>>> wrote:
>>>>> > While it is supposed to be pithy, it is also serious.
>>>>> >
>>>>> > When I wrote it, I imagine that people would do something like this:
>>>>> >
>>>>> > #lang racket/base
>>>>> > (require web-server/templates)
>>>>> >
>>>>> > (define (template-content t x)
>>>>> >   (eval #`(let ([x #,x]) (include-template #,t))))
>>>>> >
>>>>> > (template-content "t.txt" 5)
>>>>> > (template-content "t.txt" 6)
>>>>> >
>>>>> > and trust that the template wouldn't include @(system "rm -fr /"). I
>>>>> > think that a simple eval works just fine for when you, the
>>>>> programmer,
>>>>> > want to change the content dynamic (although I'd say it is better to
>>>>> > use the web-server language so you can restart with the same
>>>>> > continuations, etc.)
>>>>> >
>>>>> > Your solution is great for a template that a user provides, although
>>>>> > it has the hole that the template could call @include-template and
>>>>> > maybe find itself and go into a infinite loop, so it's not totally
>>>>> > "secure" unless you use a sandbox.
>>>>> >
>>>>> > Jay
>>>>> >
>>>>> > On Wed, Jun 5, 2013 at 1:53 PM, Joe Gibbs Politz <joe at cs.brown.edu>
>>>>> wrote:
>>>>> >> I'm writing a web server in a #lang that doesn't support macros or
>>>>> >> syntactic abstraction.  I would like to be able to use some sort of
>>>>> >> templating dynamically with bindings given by values, e.g.
>>>>> >>
>>>>> >> template.render("page-with-hole-for-username.html", { username:
>>>>> "Joe" })
>>>>> >>
>>>>> >> The FAQ (
>>>>> http://docs.racket-lang.org/web-server/faq.html#(part._.How_do_.I_use_templates__dynamically__)
>>>>> )
>>>>> >> in the documentation for templates ends with this pithy
>>>>> >> recommendation:
>>>>> >>
>>>>> >> "If you insist on dynamicism, there is always eval."
>>>>> >>
>>>>> >> I assume this isn't intended seriously, and instead to discourage
>>>>> what
>>>>> >> I'm asking about, but I'm unfortunately not seeing another way to
>>>>> >> write my program.  The code at the end of this message seems like
>>>>> the
>>>>> >> best I can do while being relatively safe, but also like a dubious
>>>>> >> inclusion on a web server given the eval.
>>>>> >>
>>>>> >> Is this the best I can hope for?  Is there another collection I
>>>>> should
>>>>> >> be looking into?  Any other recommendations?
>>>>> >>
>>>>> >> Thanks!
>>>>> >> Joe P.
>>>>> >>
>>>>> >>
>>>>> >> #lang racket/base
>>>>> >>
>>>>> >> (require
>>>>> >>   web-server/templates)
>>>>> >>
>>>>> >> (define (render-template filename dict)
>>>>> >>   (define namespace-for-template (make-empty-namespace))
>>>>> >>   (namespace-attach-module (current-namespace) 'web-server/templates
>>>>> >> namespace-for-template)
>>>>> >>   (hash-map dict
>>>>> >>     (lambda (key value)
>>>>> >>       (define name-of-identifier (string->symbol key))
>>>>> >>       (namespace-set-variable-value!
>>>>> >>         name-of-identifier
>>>>> >>         value
>>>>> >>         #f
>>>>> >>         namespace-for-template)))
>>>>> >>   (parameterize [(current-namespace namespace-for-template)]
>>>>> >>     (namespace-require 'web-server/templates))
>>>>> >>   (define to-eval #`(include-template #,(datum->syntax
>>>>> >> #'render-template filename)))
>>>>> >>   (eval to-eval namespace-for-template))
>>>>> >> ____________________
>>>>> >>   Racket Users list:
>>>>> >>   http://lists.racket-lang.org/users
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Jay McCarthy <jay at cs.byu.edu>
>>>>> > Assistant Professor / Brigham Young University
>>>>> > http://faculty.cs.byu.edu/~jay
>>>>> >
>>>>> > "The glory of God is Intelligence" - D&C 93
>>>>> > ____________________
>>>>> >   Racket Users list:
>>>>> >   http://lists.racket-lang.org/users
>>>>> ____________________
>>>>>   Racket Users list:
>>>>>   http://lists.racket-lang.org/users
>>>>>
>>>>
>>>>
>>>> ____________________
>>>>   Racket Users list:
>>>>   http://lists.racket-lang.org/users
>>>>
>>>>
>>>
>>
>> ____________________
>>   Racket Users list:
>>   http://lists.racket-lang.org/users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/users/archive/attachments/20131014/5e866dc3/attachment-0001.html>