[racket] stream-cons from racket/stream isn't lazy

An hour and a half ago, Eugene Toder wrote:
> > (That's something to know to do the code splitting that Matthias
> > is talking about.)
> Going back to this, I often find that I want to use laziness as an
> interface. That is, I build a system as a collection of components,
> written in a strict language because of familiarity/performance/ease
> of reasoning (especially in non-pure language)/etc, which are glued
> with lazy streams to make calculations demand-driven. This is known
> as generators and I think is a popular simple model.  In this case I
> don't want any module to live completely in the lazy world, I only
> want a lazy interface.

Yeah, that sounds right.  As a sidenote, the purpose of the eventual
stream thing is that once you have your lazy lists you'll be able to
iterate over them with `for' and friends.


> > Right -- on the strict side you'd want to use forces when needed.
> As an aside, why strict take has arguments in reversed order
> compared to lazy take?

The whole thing started as an experiment for a PL class, and I
followed Haskell's order -- this was before the srfi-1-order `take'
was added to the core.

> > The internal representation of sequences should change[...]
> Sequence protocol as exposed by make-do-sequence is probably the
> most complicated sequence protocol I've seen. Is it documented
> somewhere why standard sequence/iterator divide is not used?

(Yes, it's pretty complicated, I think that Matthew started with
srfi-42.)

Actually, there's a good point here, which I think was lost in the
noise of the other issues in this discussion...  If we follow this,
then:

* There is no point in the racket/stream as a library for manipulating
  iterators at all (the state makes it confusing anyway).  So as it
  currently stands, it should be removed.

* Instead, it should manipulate a new kind of a `stream' thing.

* Using one of the sequence forms (the `in-*' things) should generally
  evaluate to a stream, which `for' can iterate over.  But using it
  directly in a `for' can take a shortcut: avoid the stream, and
  produce an iterator directly.  (As far as the stream library goes,
  this means that it shouldn't really be removed, since it will exist
  there eventually.)

* One question that should now open is what should happen with
  iterators that are "very stateful" when used outside of a `for'.
  Things like `in-lines'.  One option is to just make them throw an
  error, another one is to add warnings in the documentation saying
  that doing so can lead to holding on to the file in memory.

* Another question is whether there's something that could be done
  with sequences that are much faster as stateful iterators, like
  numeric ranges.  It might mean that in such cases there's actually
  three kinds of "sequences":
  - (for ([i (in-naturals)]) ...) translates to a (fast) direct loop
  - (in-naturals) can be used as the usual lazy stream (where again,
    avoiding memoization is a plus)
  - (define nats (in-naturals)) (for ([i nats]) ...)
    In this case a sophisticated implementation can have (in addition
    to the stream value) an ability to produce a state-based producer,
    which is faster than walking the lazy stream.  (But of course not
    as fast as the first option.)  (And still should be faster than
    the current implementation of a sequence value.)

Either way, it clarifies how to deal with the state.  (Which is why
I'm surprised that the point didn't come up so far.)


> In that case sequence is stateless (or memoized) and iterator is
> statefull, but usually short-lived. Also, each sequence can provide
> multiple iterators, such as reverse or memoizing. The latter allows
> programmer to decide what he wants in a particular case.

Unless I'm still missing something, I don't see any point for
memoizing the iterator.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!