[plt-scheme] composable continuations and dynamic state

[By the way, thanks for cc'ing me -- I forgot to mention in my first
message that I am not subscribed to the list.]

   Date: Thu, 04 Jun 2009 02:04:43 -0400
   From: Ryan Culpepper <ryanc at ccs.neu.edu>

   The thread cell indirection is, as you surmised, totally irrelevant to 
   the results. You're seeing the effects of another indirection, the 
   "parameterization". Parameter-value associations are not stored 
   individually in the continuation. Instead, the continuation contains 
   "parameterizations", or mappings of all parameters to their current 
   values (really, to their currently associated thread-cells). 
   Consequently, when you separate a continuation into parts, you do *not* 
   separate the parameter updates belonging to those parts.

OK, thanks: now I understand the mechanism yielding the output shown.
My next question is about the rationale:  Why are DYNAMIC-WIND state
points only partially preserved, whereas parametrizations are fully
preserved?

I did a couple more tests, with some surprising results, showing that
`fully preserved' is not quite right in the previous question, and
actually I mean `fully preserved or not preserved at all'.  The first
test is to compare the values of the following two expressions:

(let ((parameter (make-parameter 'foo))
      (prompt-tag (make-continuation-prompt-tag)))
  (let ((f
         (parameterize ((parameter 'bar))
           (call-with-continuation-prompt
             (lambda ()
               (call-with-composable-continuation
                   (lambda (continuation)
                     (abort-current-continuation prompt-tag
                       (lambda () continuation)))
                 prompt-tag)
               (parameter))
             prompt-tag))))
    (f)))
;Value: foo

(let ((parameter (make-parameter 'foo))
      (prompt-tag (make-continuation-prompt-tag)))
  (let ((f
         (parameterize ((parameter 'bar))
           (call-with-continuation-prompt
             (lambda ()
               ;** Here is the only difference.
               (parameterize (((make-parameter 'mumble) 'frotz))
                 (call-with-composable-continuation
                     (lambda (continuation)
                       (abort-current-continuation prompt-tag
                         (lambda () continuation)))
                   prompt-tag)
                 (parameter)))
             prompt-tag))))
    (f)))
;Value: bar

In lieu of the way parametrizations are implemented, I think I
understand the mechanism that yields this result (simply putting a
parametrization in the continuation of CWCompC has the effect of
saving all the parameters in the composable continuation), but I don't
think that makes the high-level behaviour any less surprising.

Now, the current semantics of composable continuations and dynamic
state suggest to me that a naive user (such as me) might attempt to
mix DYNAMIC-WIND and PARAMETRIZE with disastrous results.  I don't
have a good example of this right now, but here's a pathological one
to keep you occupied.  Consider a naive recursive mutex in terms of a
semaphore and a parameter that tells us whether we already have the
mutex locked:

(define-struct recursive-mutex (semaphore parameter))

(define (new-recursive-mutex)
  (make-recursive-mutex (make-semaphore 1)
                        (make-parameter #f)))

(define (with-recursive-mutex-locked recursive-mutex thunk)
  (let ((parameter (recursive-mutex-parameter recursive-mutex)))
    (if (parameter)          ; We own the mutex already.
        (thunk)
        (let ((semaphore (recursive-mutex-semaphore recursive-mutex)))
          (dynamic-wind
           (lambda ()
             (semaphore-wait semaphore))
           (lambda ()
             (parametrize ((parameter #t))
               (thunk)))
           (lambda ()
             (semaphore-post semaphore)))))))

We can check whether we actually have it locked by asking the
semaphore whether it agrees, using SEMAPHORE-TRY-WAIT? like so:

(let ((recursive-mutex (new-recursive-mutex)))
  (define (test)
    (write (let ((semaphore (recursive-mutex-semaphore recursive-mutex)))
             (if (semaphore-try-wait? semaphore)
                 (begin
                   (semaphore-post semaphore)
                   'no-we-doesnt)
                 'we-has-it-my-precious)))
    (newline))
  (with-recursive-mutex-locked recursive-mutex test)
  (test))

As expected, the output is:

we-has-it-my-precious
no-we-doesnt

Now consider the following program using recursive mutexes, and also
SEMAPHORE-TRY-WAIT? to discern whether we actually have the mutex
locked:

(let ((recursive-mutex (new-recursive-mutex))
      (prompt-tag (make-continuation-prompt-tag)))
  (let ((f
         (with-recursive-mutex-locked recursive-mutex
           (lambda ()
             (call-with-continuation-prompt
               (lambda ()
                 (parameterize (((make-parameter 'foo) 'bar))
                   (call-with-composable-continuation
                       (lambda (continuation)
                         (abort-current-continuation prompt-tag
                                                     (lambda ()
                                                       continuation)))
                     prompt-tag)
                   (with-recursive-mutex-locked recursive-mutex
                     (lambda ()
                       (write
                        (let ((semaphore
                               (recursive-mutex-semaphore recursive-mutex)))
                          (if (semaphore-try-wait? semaphore)
                              (begin (semaphore-post semaphore)
                                     'no-we-doesnt)
                              'we-has-it-my-precious)))
                       (newline)))))
               prompt-tag)))))
    (f)))

In the call to F at the end, we don't have the recursive mutex locked,
and if we call F, it will not attempt to lock the recursive mutex
because the composable continuation did not capture the DYNAMIC-WIND
in-thunk to do that.  But F will nevertheless have the recursive
mutex's parameter bound to #T (i.e., it will have continuation marks
with a parametrization that maps the recursive mutex's parameter to a
thread cell whose value in the current thread is #T), making it think
that we have the recursive mutex locked, so that it will not wait for
the semaphore when it tries to lock the recursive mutex.

Using CALL-WITH-SEMAPHORE rather than DYNAMIC-WIND, SEMAPHORE-WAIT,
and SEMAPHORE-POST, has the same effect, by the way.  In either case,
the output indicates that F failed to lock the mutex.

Also, you may observe the curious (parametrize (((make-parameter 'foo)
'bar)) ...) construction.  If you take that out, F succeeds in locking
the mutex.