[plt-scheme] Re: Using LibFFI along with threads

On Jul 17, Hans Oesterholt-Dijkema wrote:
> I'm sorry. It may be a matter of taste, but I don't like this
> solution.
> 
> Also, starting a starting and synchronizing a thread is usually way
> faster than starting a subprocess. Kernels typically have special
> logic to support threads these days.

First of all, I have to say that I'm *very* aware of costs...  It will
be hard to find me write code that spends CPU cycles where they can be
avoided.  But of course there are factors that can be worth the CPU
cycles -- if it makes it easier to develop, extend, and maintain.
(And there are plenty of these in PLT.)

In this specific case, when I consider all the factors (those that I'm
aware of, obviously), I see an overwhelming advantage for the solution
I suggested:

* First of all: it's all Scheme code.  This is a major benefit in
  almost every aspect.

* It's portable across all platforms.  No need to deal with various
  Linux thread packages, which is probably worse when dealing with
  cases like BSD or Solaris.

* It's easy to deal with state.  For example, you need to authenticate
  to the DB, run a query, then disconnect -- you can do all that in a
  single such sub-process.  No complications.

* Another example: take Noel's suggestion of letting the user control
  thread allocation and thread use -- with the Scheme code it's all
  very easy to achieve.  It's also easy to keep a subprocess on hold
  after it's done, and re-use it next time you need an async
  operation.  My guess is that doing all that in C will be much more
  painful.  BTW, I looked at your code -- and it looks like you're
  starting a new thread for each async call, so if you have several of
  these, then it's likely to be more expensive than starting a single
  subprocess.

* It's a pretty simple interface, compared to everything that you can
  do with threads.  (Related disadvantage: you cannot do everything
  that you can do with threads.)

* The only major disadvantage that I can think of is the cost of
  starting a process (which in this case includes the cost of booting
  mzscheme).  But:
  - if you keep such subprocess alive, then you don't pay that cost
    for every async use
  - you say that the operation is heavy enough that you don't want the
    GUI to be frozen while it is done -- and I think if this is the
    case then you have much higher costs for the DB operation itself,
    so the subprocess would not be a significant (or even noticeable)
    contribution.

* (BTW: On sane OSs, forking a process is almost as cheap as running a
  thread -- and that's pretty close to the external process approach.)

I do think that there is a value in making async calls that use
threads.  But I'm not convince that this is the case with the problem
you describe.


> Did you have a look at the spike I sent to the list? It missed the
> code in c-threads.c, but that can be found in the sqld-psql-c.plt
> package.

I skimmed through it.  IIUC, it is better done as a flag for the
actual call-out, and the interface from Scheme is basically the same
except that Scheme threads will continue to run as usual.  So instead
of a callback or a sync event you just do the foreign code in a Scheme
thread.  This sounds (again, IIUC) like a good approach.  The first
thing to do is to be very sure that your `c-threads' code works fine
(and behaves the same) on all platforms.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                  http://www.barzilay.org/                 Maze is Life!