[plt-dev] Test server & client

On May 11, Matthias Felleisen wrote:
> 
> On May 11, 2009, at 6:21 AM, Eli Barzilay wrote:
> 
> > but this was an extreme example of "live coding", much more
> > stressful than what's usually associated with that term...
> 
> To wit, it was so live that Eli was still writing the test while the
> students were taking it :-]

I forgot to include the important bits of the README file, see below.
One of the advantages I listed for doing this is that you can come to
the exam with only a sketch for some question, and write them fully as
the exam goes...  In my case, I was busy hacking on the server and
client for the first ~1-2 hours, and then I wrote two question.  It
was fun.


----------------------------------------------------------------------

This is a server and a client to run a paperless test.  The clients
will run in "kiosk mode" -- locking out all other applications, which
is intended to prevent students from chatting, looking up stuff on the
web etc.  This is currently working only on Windows.  The server can
run in any platform, but it is slightly more convenient if it is
unix-like (linux or osx), preferably inside Emacs.


Why?
----

Running a paperless exam has some major advantages:

* When you get back from your exam you don't need to schlep a pile of
  papers.  You don't need to count them every five seconds to make
  sure that they're all there.  And you also don't need to worry about
  leaving the pile of graded exams on the roof of your car and realize
  later that you forgot them there, then spend three hours scanning
  the roads and sidewalks.

* You don't need to deal with student handwriting (sometime even
  intentionally obscured -- I've seen cases where students who didn't
  know the answer wrote text that could barely be read).

* You can provide the students with additional materials, so they
  don't need to kill half of a tree because your exam is open
  materials.  (However, this is limited to text files.)

* You can update files during the test: clarify questions, add hints
  etc.  You can even have a rough sketch of some questions and write
  them up as the test goes on...

* You can provide "template files" for the students to modify.  For
  example, you can give them some function and ask to fix a bug.

* Grading is often a repetitive job, and I found myself starting with
  writing detailed comments, and later on (when the 10th exam has the
  same mistake) settle on an explanation-less cross and other obscure
  markings.  When you edit files, it is easy to copy+paste the same
  comment.

* Grading can be more consistent if you're doing such copy+paste that
  includes the grading.  In addition, it is easy to grep for a
  particular error that you've seen to make sure that the grading is
  consistent.

* Obviously, you can easily come up with a textual markup system so
  that you compute the exam grades automatically from the graded
  files.  I'm using a system like that for homework submission and
  grading, and having exams in electronic formats fits nicely into
  this system.

* The same holds for distributing the works.  You can keep the
  original pristine files in case students want to complain -- no need
  to worry about pencil-written exam being modified post-grading, and
  you can have the graded files available through your usual channels
  (eg, through the PLT handin-server).


Features
--------

The server and the client were designed to be *very* robust and
secure, while being convenient to use.

* Server-client communication is done through an encrypted connection.
  The server can be setup to accept connections only from some known
  IP addresses (and will report attempted connections from other IPs).

* Clients do not have a "save" or a "submit" button -- the text is
  continuously sent back to the server for storage.  This frees
  students from dealing with such details while they're stressed for
  time.

* Clients do *not* have a Scheme REPL.  This wouldn't be too hard to
  add, but I think that this would be bad for several reasons:
  students can easily get dragged to a long debugging session instead
  of focusing on the test; you can get dragged to asking tougher
  questions since the students can now actually try things out; and
  you can get dragged to harsher grading ("if it doesn't work, you
  don't get the points", versus "you got the rough idea right, so you
  get the points" which is common with written exams and make them
  less stressful).

* The server backs up the directories of all clients periodically, to
  make things safer, and to avoid "I accidentally hit delete in the
  last minute" excuses.

* Clients use a number of tricks to make the application run in "kiosk
  mode": making it nearly impossible to switch to another application
  or kill the client.  There are cases when some of these may fail
  (for example, at the NEU lab ctrl+alt+delete cannot be disabled) --
  the client has a safety mechanism against this: if you hit
  ctrl+alt+delete it will notice that it's not the focused window, and
  then it will (a) alert the server, and (b) set itself back into
  focus.

* You can set up a master password that can unlock a client.  (Hit
  ctrl+shift+P, then the password, then enter.)  This is another
  safety feature in case of some server catastrophe: in this case you
  can manually unlock a station and copy the written solution out to a
  different application.

* The server keeps a live connection with all clients and polls them
  frequently.  This is done for two reasons:
  - if the client dies (either due to a bug, or to a break attempt),
    the server will detect and report it,
  - on each poll the client reports sends the server the updated
    contents of editable buffers to save.

* The client can withstand being disconnected from the server (eg, if
  the server is restarted for some reason) -- it will alert the
  student, but will continue to operate as usual.  (Specifically, the
  student can continue reading files and writing answers during such
  downtimes.)  Changes are not lost -- so when a connection is
  re-established, synchronization continues as usual.

* You can update client-visible files on the server -- the changes
  will be sent to all clients, and the clients will visually indicate
  modified files so students can easily see that they need to read
  something.

* The server can require users to login to be able to edit files.  In
  this case, the files that the student writes will be saved in a
  directory with the username.  It can also work in a login-less mode,
  so each client corresponds to a single student working; in this
  case, you will need to make students write their names in their
  answer file(s).  (This sounds insecure, but works perfectly fine in
  practice: as long as a responsible person is present in the room,
  making sure that people do not switch stations.)  You can also
  require logins without having a password file: and login students on
  the machine using the master password which will work with any
  username, and you can even do so remotely from the server prompt.
  (This is also useful if some students forgot their password.)

* If the server is set up to require logins, the client is still
  functional before such login is successful -- but it will not show
  editable files.  This is useful, for example, if you have twice the
  number of machines than the number of students: each student can
  then use one machine to write their answers, and another machine to
  read the exam material.

* Clients are identified with a unique client id (which you setup when
  the clients are started, or leave the default as the IP number), and
  after logging in, they are identified by usernames.  These must be
  unique, so no user can login on two clients at the same time (but if
  a machine crashes, it is possible to have the student move to a
  different machine).  In our lab, I set each client id to correspond
  to the physical location of the machine, so it is later possible to
  know if two students were sitting next to each other.  (When there's
  suspected copying, for example.)

* If a client is disconnected, and later reconnected, it will show the
  same content, including files that were modified by the logged-in
  username (if logins are required) or files that were modified on the
  client with the same client id -- so no edits (beyond the few
  seconds of poll frequency) are lost on a client crash.

* The server controller can send messages to all clients.  The message
  will flash at the bottom of each client to avoid a student missing
  it.  The message is also logged in a "messages" buffer, so students
  can read these messages later again.  The controller can also send a
  message to a single client.

* Clients can send a message to the server, which is done with a
  chat-like interface.  Together with the above, it is possible for
  students to ask you questions privately, and for you to announce a
  message back for the whole class.  (For example, a student asks a
  question, you realize that further clarification is important so you
  edit a file or add a new one, and tell everyone about it.)  It is
  also possible to disable this feature in the configuration file.

* The server controller has extensive control over the clients through
  several commands (more below).  It is possible to freeze all clients
  (in case you want to make sure that students listen to what you're
  saying) or a specific one; force all clients (or a specific one) to
  switch to the messages window (in case you want to make sure they're
  reading some messages that you're sending); unlock a client (or all)
  remotely (similar use case as the master password); restart a client
  (or all) (eg, to recycle the machine for someone else); or kill a
  client (or all) (which is useful at the end of the test).

* The clients can show the time -- which you can configure to be a
  normal clock, or a countdown to a specific time.  You can also
  change this time at the server prompt, including to any random
  (short) text (for example "1min!").  In all cases, the displayed
  time is controlled by the server, so a station with a bad internal
  clock will not cause problems.

* Many more aspects of both the server and the clients are
  customizable.  All the way down to indicator colors.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                  http://www.barzilay.org/                 Maze is Life!