[plt-scheme] Exporting from an already-written module?

On Jan 19, Don Blaheta wrote:
> Quoth Eli Barzilay:
> > See also the reply I sent last evening describing how the sandbox
> > library can be used for such things.  (Given that you already have a
> > bunch of files and only want to script checking them, the sandbox
> > library is what you really need, not the handin server.)
> 
> Yes, this is almost exactly what I was looking for (and for those
> reading this via archive, the relevant post is in the thread "module
> access from test cases file", partially quoted below).  I did have a
> problem with the suggested incantation:
> 
> >  (require scheme/sandbox)
> >  (define e (make-module-evaluator (string->path "...your file...")))
> >  (e '...some-expression...)
> 
> when I did this, I got some errors from open-input-file claiming
> that it lacked read permission (though the file was in the current
> directory and readable);

Ah yes, I forgot that this was one of the things that I fixed after
4.1.3 came out.  It will work fine with 4.1.4 which will be out
shortly.  (The problem is not with the file being readable to you,
it's about the sandboxed environment being very limited in what it can
access.  The fix I did was to allow it to read the specified path when
one was specified as the source of a module-based sandbox.)


> reading the doc for make-module-evaluator suggested that it could
> take a port instead, so I tried
> 
>   (define e (make-module-evaluator (open-input-file (string->path "...your file..."))))
> 
> explicitly and this worked;

[That's a good workaround for now.]


> I can now evaluate expressions in the context of my students'
> definitions.  I'm still not sure why the first version failed,
> though.

I hope that the above clarifies it.  Generally speaking, the safe
sandbox is especially important in a case like checking homework
submissions, since you can never tell when some student learns enough
stuff to hack your system.  (For example, back in the very early days
of the code, a student could submit code with something like:

  (error (file->string "/etc/passwd"))

to the handin server, which would happily report the error back to the
submitting client...)


> Now I'm on to actually writing the test cases.  Is there any way to
> force a local on the evaluator?  That is, if I do
> 
>   (e '(define 'foo 3))
> 
> then this definition is permanent, and I'm not seeing a way to roll this
> back short of making a new evaluator.  Is there a way?  (On reflection,
> I don't really need this for the test cases I'll be doing, but I'm still
> curious if it can be done.)

I'm not sure exactly what you're trying to do here...  In general,
these evaluations happen just like they do on a repl, so the above is
similar to entering the exact expression in the repl, which -- with
the way Scheme reads a quote -- simply redefines `quote' as a
function.

I suspect that you want to define your own binding for some value that
comes from the student code.  You can then use `quasiquote' and
`unquote' to use these values.  For example, say that the student file
looks like (I'm using the `scheme' language for simplicity):

  #lang scheme
  (define foo (* 123 456))
  (define (add x y) (+ x y))

Then you can bind your own `x' to the twice the value of the student's
`foo':

  (define x (* 2 (e 'foo)))

and use this value with the student's `add' function:

  (e `(add foo ,x))

Using the sandbox environment is very convenient, because it provides
a clear separation between what happens inside the student code and in
your testing code.


> Relatedly, since the evaluator is now "in" one of the HtDP
> languages, I don't have access to e.g. namespace-defined?,

This problem is unrelated to the below.  You basically want to check
that a name is defined in the student's code, so you want to use
`namespace-defined?' in the sandbox -- but you can't do so for the
obvious reason.  That's a good thing, since you don't want the htdp
languages carrying a bunch of such functions just to be able to test
the code.

In any case, the solution for this is a new `call-in-sandbox-context'
function which will run an arbitrary thunk (*your* code) inside the
sandbox environment.  For example, to check if a name is defined using
that function, you can do this:

  (require mzlib/etc)
  (call-in-sandbox-context e (lambda () (namespace-defined? 'foo)))

(This is also one of the new additions, so you'll need a nightly build
until v4.1.4 comes out.)


> and the student code can in any case crash or not work, and this
> appears to take down the whole program, sandboxing notwithstanding.
> For instance, if I
> 
>   (e '(/ 1 0))
>   (printf "Testing.~n")
> 
> the printf never executes.  Am I misunderstanding the role of the
> sandbox here?  I think I can work around this the same way I used
> to, with a with-handlers clause that catches exn:fail?, but I
> suppose I was expecting that the sandbox would take care of some of
> that for me.

Well, the sandbox detects the error and it needs to do something with
it.  The obvious thing to do is to simply propagate the error up to
the calling code, and let you catch it with a `with-handlers'.
Otherwise, it would need to somehow give you back a different kind of
a value to indicate a raised exception.  The same holds for multiple
values -- given a sandbox with a `values' binding (like testing code
in the `scheme' language), a test like:

  (equal? 123 (e '(blah)))

might throw an error if `blah' returns multiple values -- the same
kind of error you'd get with

  (equal? 123 (values 1 2))

Therefore, uses of sandboxes almost always need to catch all
exceptions from the sandbox, and be aware of multiple values.  But you
do get a lot of help in that *all* errors are just exceptions.  For
example, entering any of these will create errors that are hard to
catch otherwise:

  )
    that's not a typo -- this should be a read error.

  (if (< 1 2))
    you won't be able to put this in a test expression, since the
    expression would not compile right -- you'd get a syntax error in
    the test itself.

  (letrec ([loop (lambda () (loop))]) (loop))
    this will not work right in a simple test for the obvious reason.

  (kill-thread (current-thread))
  (exit)
    these won't be something you can test since the expression will
    just kill the whole thread or exit mzscheme.

Having said that, perhaps it makes sense to provide some wrapper that
returns one of

  (list 'values <resulting-values>)
  (list 'raise <some-raised-value>)

I didn't do that since it seems like a trivial thing to setup.  (By
the way, note that there can be anything in the second form, since you
can call `raise' directly on any value, not just an exception.)


> Finally, I tried poking around with the new HtDP-style test cases, per
> Matthias's suggestion:
> 
> Quoth Matthias Felleisen:
> > #lang scheme
> > (require htdp/testing)
> > 
> > ...
> > 
> > (generate-report)
> > ;; the end
> > 
> > is close enough.
> 
> This was somewhat helpful, although I can't find any documentation
> on it.  I did find a post from Kathy Gray from last September that
> mentioned that htdp/testing was deprecated, apparently in favour of
> test-engine/scheme-tests, which also seems to work; but I can't find
> any docs on that either....  It looks like it won't be quite what I
> want, because check-expect seems to need to be at the top-level, but
> I'm not sure if there's a workaround.

I can't help with this, since I'm not sure about the details.  I do
remember some issues with the teaching languages tests when it comes
to automating things with the sandbox, but I don't remember if there
was some simple solution.  (It involves some obscure details that make
doing the right thing outside of DrScheme very difficult.)

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                  http://www.barzilay.org/                 Maze is Life!