[racket] Embedding multiline shell scipts

Thomas Chust wrote at 02/06/2011 10:14 AM:
> 2011/2/6 Manfred Lotz <manfred.lotz at arcor.de>:
>   
>> [...]
>> I don't know how to merge stdout and stderr in proper sequence.
>> [...]
>>     
>
> There is no such thing as proper sequencing of data flowing through different streams.
>
> Due to buffering in system libraries and the operating system kernel it is also entirely impossible to reliably process the output data in the exact sequence it was produced by the external process.

Agreed with Thomas that doing this perfectly is impossible in the 
general case, even when there's a single-threaded producer of the two 
streams.

However, if you want to sequence reading stdout and stderr close enough 
for most purposes, keep in mind that these are usually interleaved at 
the resolution of lines rather than characters, so you can get this 
pretty much right, and still do fairly efficient block reads.  So, you 
can do the sequencing efficiently in your reading process by doing raw 
stdio and avoiding any buffering port abstractions in your library, 
doing a "sync" loop on the ports, and implementing your own buffering 
for each port, out of which buffer you pick a complete line at a time.  
You also can tweak this to better interleave reads from the two 
streams/buffers.  You can also use time or other heuristics to detect 
when, say, a stdout message like "Processing..." without newline should 
be handled because it was interrupted by a subsequent stderr error 
message.  There might still be buffering outside of your control, but I 
think it doesn't often matter.

Once you get into "expect"-like interaction with a process using the 
three streams at once, and doing pattern-matching while also capturing 
the output, and doing this all efficiently, it's tricky.  There's no 
reason that a generalized library routine to do this can't be 
implemented in Racket, but I'm not aware of one.  If you hand-implement 
for a particular purpose, I think you'll be getting your hands dirty 
with "sync", your own buffering of both input and output, block I/O, and 
some strategy for how to do your pattern-matching efficiently on the 
buffers.  IMHO, this should feel like systems programming if you're 
doing it efficiently.  This complication does not reflect a limitation 
of Racket, but instead that Racket is flexible enough that this doing 
this complicated stuff is possible.

-- 
http://www.neilvandyke.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/users/archive/attachments/20110206/8f0fa856/attachment.html>