[plt-scheme] Web framework question

On Sun, Apr 11, 2010 at 6:17 AM, Johan Coppieters <johan577 at mac.com> wrote:
> Jay,
>
> Nice and clear answers!
>
> To all interested in Scheme and/or web development. Any reaction will help.
>
> I'm building web-apps as a day to day job. We've started a company almost 15 years ago and are now among the 10 biggest web-builders in Belgium. At that time Netscape commerce server was the only option, but it was too expensive for a small startup, so I've created our own application engine/language, still being used by most of our programmers and they are happy with it. It's highly performant and let's you develop very rapidly.
>
> But now it is time to move on...  We need a new environment... So I'm trying...
>
> MVC / object oriented is without a question. (I come from a time where Smalltalk emerged and where I've spend nights behind a Symbolic lisp machine)
> Persistent data should be stored in an SQL database, but it should be wisely used, not as most PHP/CMS solutions like Drupal and others that execute a multitude of queries for every page they serve. We have clients for which we serve more than a terrabyte of html web app data per month.
>
> I've been experimenting with a number of alternatives. After 6 months, I'm left with 2 possibilities:
> - Go mainstream this time, pick java/jsp, it's robust, widely used and well suited for MVC and you can find many programmers already experienced with the tools.
> - Go rogue again and build a framework on top of the Scheme web-server.
>
> I've converted part of a CMS into Java/Jsp as well as in Scheme/Templates. Both went very well. A number of things crossed my mind:
>
> 1) I'm not sure using continuations is the best way to keep sessions for users.
>
> We've always built our solutions with "web" in mind and never tried to implement them as you would do with a traditional program.
> Web is web, I don't believe you should try to mimic another behavior into it. Except for login data, language choice and other minor things. Trying to put a real flow into it, is always looking for trouble. People tend to hit back buttons multiple times, use the history menu's, bookmark page or leave a webpage open for hours, if not days, before continuing. You can't (and shouldn't) change the nature of a web application. It should work with request-response whenever possible.

Continuations and functional programming will work "naturally" with
the back button, etc. That was part of the original point.

It's normally not very useful to think of a continuation as a
"session". If there is a part of your Web application which is a
multiple step computation involving the user, then using continuations
is the best way to write it, because send/suspend lets you hide the
interaction in your code and program the direct-style computation.
Examples: checking out a shopping cart. Non-examples: Browsing a list
of options. [You can pretend that the computation is "Selecting an
option" and the user is part of that process, but this is not that
much more useful than traditional style in practice.]

> 2) Some of our clients have very busy sites/web-apps/intranets. We need to be able to upgrade these applications live, without breaking their sessions. So the session data needs to be serializable and it should be possible to have more/different session data after the upgrade.

Continuations are not upgradable. Serializable continuations are
serializable. =)

I normally structure things as follows:

- The "computations" are assumed to be short-lived, so I use the
continuation functionality.

- Everything else is assumed to be long-lived, so I write a few
web-server/dispatch custom arguments that serialize parts of the
user's data into the URL and then write a dispatch-rules that
structures the URLs. These will be upgradable, persistent, etc.

I used to use continuations for everything. The only difference
between then and now is that I used to write:

(a ([href ,(embed/url top-level-function arg)]) ...)

in the context of send/suspend/dispatch

Now I write:

(a ([href ,(dispatch-url top-level-function arg)]) ...)

where at the top level I wrote:

(define-values (dispatch-request dispatch-url)
 (dispatch-rules
  ...
  [(some url pattern) top-level-function]
  ...))

Another perspective on this: if the continuation of a call to
send/suspend/dispatch is the top-level continuation and the handler is
top-level, then you can convert to dispatch-url.

I should probably write a dispatch-arg pattern that puts a closure in
the manager, so that you could use dispatch-url easier with
non-top-level handlers.

>
> 3) The language used in the Model-Controller part should be the same (or at least integrate seamlessly into) the HTML pages that are used as View. The html pages should be in (x)HTML, because also designers (slicing and generating CSS), will need to touch and modify these pages from time to time. These HTML pages need on a development platfom to be easily changeable. If you've ever seen someone changing stylesheets, you know they update/save/test the page at least 10 times per minute...
>

Using 'eval' with include-template is probably most useful for this.

> 4) We need some kind of AMVC model. I've added a A for Application because some data and methods need to be on application level (multi language label data, navigational model, etc...) . There is no need to fetch this data for each request and/or session. I don't want an IO / DB trashing framework.

Nothing about PLT helps or hurts this part of your job.

Jay

>
> Any advise or thoughts are more than welcome...
> Johan.
>
>
>> Simple answer: Templates are compiled into your application. Thus, you
>> Need to recompile and restart.
>>
>> Long answer 1: Why are they compiled in?
>>
>> Since templates can include arbitrary Scheme code and refer to
>> arbitrary identifiers, an 'include-template' is *really* just a funny
>> 'require' statement.
>>
>> Long answer 2: Why do you need to restart your server when you recompile?
>>
>> You are using serve/servlet and passing it the closure named
>> 'render-blog'. If the code that creates that closure changes, there's
>> no way to know that and update it.
>>
>> Long answer 3: What if I didn't use serve/servlet?
>>
>> However, even if you were using a filesystem-based servlet-dispatcher,
>> it still wouldn't do what you might think when you run
>> 'conf/refresh-servlets' (which deletes the compiled code cache and
>> thus finds new closures the next time you go to your app); that is if
>> you expect it to update the template for continuations you've already
>> captured. That's because the captured continuations are like closures,
>> so they refer to the old data from the original code not the new data
>> from the recompiled code. It is not possible to 'port the
>> continuation' to change these references because the new code can be
>> arbitrarily different.
>>
>> Long answer 4: What if I used serializable continuations?
>>
>> Continuations are serialized with a hash that ensures that any source
>> code modifications makes all the old continuations incompatible for
>> the same reason native continuations naturally are.
>>
>> Long answer 5: Is there any way to trick it?
>>
>> Of course. If you use a model-view-controller structure with three
>> different modules, then when you update the template you're changing
>> the view module. However, the continuations will be serialized with
>> the controller's module code hash (because you obeyed MVC well), thus
>> everything still works: good job!
>>
>> Long answer 6: Is there any way to trick it without serialized continuations?
>>
>> Of course. Put your include-template in an "eval". So that it all
>> happens at runtime anyway. This will be slower (but who cares on the
>> Web.) [I write "eval" because eval is a pain to use and something like
>> sandboxes might actually be more straight-forward.]
>>
>> Jay
>>
>> p.s. I will put something like this in the FAQ.
>> Johan  
>>
>>
>>
>>
> _________________________________________________
>  For list-related administrative tasks:
>  http://list.cs.brown.edu/mailman/listinfo/plt-scheme
>



-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://teammccarthy.org/jay

"The glory of God is Intelligence" - D&C 93