[racket-dev] using Racket to build an image-based Lisp: feasible?

On Mar 5, 2013, at 12:12 PM, Eli Barzilay <eli at barzilay.org> wrote:

> A few minutes ago, mikel evins wrote:
>> 
>> On Mar 5, 2013, at 11:55 AM, Eli Barzilay <eli at barzilay.org> wrote:
>> 
>>> An hour and a half ago, Matthias Felleisen wrote:
>>>> 
>>>> I have empathy for your perspective, because I have worked with
>>>> images and -- at the time -- found them useful.
>>> 
>>> I think that *most* of Mikel's long post was about the advantages
>>> of having a live REPL,
>> 
>> Then I didn't express myself clearly enough. Sorry about that.
> 
> So let me clarify a bit more (though this is obviously my very
> subjective opinion)...  I loved doing the kind of live-REPL-based
> debugging that you're talking about, but every once in a while things
> would get messed up in a way that made it much easier to just restart
> the whole thing.  I've also used images, but the main thing that
> bothered me about them is that they're heavily oriented in the other
> direction: instead of restarting the application from scratch where I
> know that the code is only what's in the source, it encourages
> continuous tweaking of the live REPL where you continue to tweak and
> try to fix it to later dump a new image.
> 
> Eventually, I concluded that images are a cute thing in theory, but in
> practice they were too limited in the reduced ability to deal with the
> source code.  Maybe Emacs is a good example of this: even though it's
> image-based, developement happens on the source, with the image
> feature being pushed to just another step in the compilation chain.
> 
> (Disclaimer: the systems that I used with images were the common ones
> where there's no support for editing the live source -- not the fancy
> lisp machine or smalltalk things.)

The problem you're describing is real, and is one of the prices paid for image-based development. It doesn't cancel out the advantages, but it's something you have to learn to deal with appropriately. One of the ways you deal with it is through process: we kept reference images and checkpoints, so that we could roll back troublesome changes. That wasn't burdensome precisely because saving and loading an image was so fast and easy. 

Another way you deal with it is to develop the kinds of tools you mentioned--tools that give you greater power to identify and change the things you want to change. Some of the tools that working programmers take for granted nowadays were invented in that context, and I think it's reasonable to expect that more of them will be, as long as we don't forget how to make and use image-based environments.

I had to think some more about how my remarks might have led you to think I was talking about REPLs. Let me take another stab at it by mentioning things that REPLs don't provide.

I don't in any way want to discount the value of a live REPL; it's indispensible. It's not what I was talking about, though.

By itself, a live REPL doesn't grant me the power to preserve the incremental dynamic state of my work-in-progress with a single quick action, nor to equally quickly recover that state. When I have that power, I use it constantly.

It doesn't make delivery of incremental release candidates as quick and simple as saving a word-processing document.

It doesn't offer me a bog-simple, quick, and convenient way to preserve snapshots of my progress on a task, nor to provide snapshots I can use to reproduce *exactly*, and with all its dynamic details, a pathological state.

It doesn't offer me a way to package a whole dynamic machine state to give to someone else, so that they can see what I see, nor an equally quick and simple way to illustrate proposed changes by making them directly in their intended context in a way that another person can see and interact with them immediately.

It doesn't give me a way to checkpoint the space of changes and use the checkpoints to zero in on the change that introduced a bug.

It doesn't give me a quick and simple way to roll back an unfortunate change, and without that ability, experimentation carries a higher cost. Maybe lowering the cost of experimentation seems a small advantage; it isn't.

REPLs don't do any of these things, though they complement them nicely. If you have these abilities, they're better if you also have a REPL, and the reverse is also true.

REPLs are a dime a dozen nowadays. Clojure has one. So do FORTH, erlang, Ruby, Python, the various flavors of ML, and Haskell. The REPLs in Lisp and Smalltalk systems are different. TO a much greater degree than in other systems, you can use them to incrementally build up state that provides a progressively closer approximation to what you're trying to build. An image-based Lisp or Smalltalk is better still, because you don't have to recapitulate the groundwork every time you start the REPL; instead, having achieved a congenial state, you can just save it, and then resume it later, just as easily.

Naturally, you don't always want that. Sometimes a reference state, free of accumulated deltas, is what you want. Image-based systems don't in any way prevent you from having that. On the contrary, they make it easy to provide yourself with several different ones for different uses. You do, of course, have to realize that you're going to want that, and take appropriate steps to ensure that you have it when you want it. That's one of those things that comes with experience, or with good advice from an experienced colleague.

But not having the capability to save and load an image provides no advantages. Anything you can do without that capability, you can do with it. The reverse is not true.

--me