[racket-dev] net/http-client

On Wed, Aug 28, 2013 at 3:30 PM, Greg Hendershott <greghendershott at gmail.com
> wrote:

> This looks great!!
>
> A couple suggestions:
>
> 1. Support for "Expect: 100-continue" request headers would be
> helpful, and I think not too messy to add.
>
> The big use case I'm aware of is Amazon S3. If you make a PUT or POST
> request, it might need to redirect you to another URI (outage,
> balancing, whatever reason). Expecting and handling 100-continue lets
> you avoid transmitting potentially large amount of data that would be
> discarded, and you have to send it all over again in the request to
> the redirect URI. For (say) a 1 GB upload to S3, this matters.
> Although I don't know for sure if other upload-ish web APIs offer
> same, I'd guess some do as this is the use case for 100-continue
> generally.
>
> How I implemented this in my HTTP package was to have a
> `start-request` function that sends the request line and headers,
> peeks the response status line, then returns a `boolean?` whether the
> PUT/POST/PATCH/whatever data should be transmitted. [1]
>
> I think your `http-conn-send!` could do similar?
>

Do you think it is appropriate to expect the http-client user to put in the
"Expect: 100-continue" or better to always send it if there is a data
component?


> 2. Support for "Content-Encoding" response headers would also be helpful.
>
> Using the same make-pipe approach as you're doing with chunked
> transfer encoding. [2]  Maybe this is mission creep: For HTTP 1.1. you
> _must_ support Transfer-Encoding: chunked, whereas Content-Encoding is
> just optional. However it's a good option; using compression can
> really help out on time as well as bandwidth charges.
>

I just pushed support for this.


> IIRC those were the two main things that motivated me to make my HTTP
> package at all, to support e.g. my AWS package. If http-client added
> them, I might not need my package anymore. (OK, it might take me
> awhile to phase it out until I'm ready to de-support older versions of
> Racket, but, I and others wouldn't need it for new projects.)
>
>
> [1]:
> https://github.com/greghendershott/http/blob/master/http/request.rkt#L142-L189
>
> [2]: By the way, do you want to pass some `limit` optional arg in the
> various uses of `make-pipe`? Otherwise IIUC this will suck everything
> into RAM, which might not be so great with very large request or
> response entities.
>

Matthew changed this a few days ago.

Jay


>
>
> On Fri, Aug 23, 2013 at 2:48 PM, Jay McCarthy <jay.mccarthy at gmail.com>
> wrote:
> > Based on a request back in early July to remove the restrictions that
> > net/url puts on HTTP communication (vis a vis URL encoding), I have
> > just pushed a new HTTP client as net/http-client.
> >
> > The push also changes net/url to use net/http-client so that we just
> > have 1 HTTP request producer, rather than 3.
> >
> > It passes all of the net/* tests, but these don't use features like
> > proxying, HTTP/1.1, etc. I'm slightly nervous that it doesn't do those
> > correct, but not super nervous, because I just cut-and-pasted the
> > code.
> >
> > The main approach of the library is best explained by this contract:
> >
> > [http-sendrecv
> >    (->* ((or/c bytes? string?) (or/c bytes? string?))
> >         (#:ssl? (or/c boolean? ssl-client-context? symbol?)
> >                 #:port (between/c 1 65535)
> >                 #:method (or/c bytes? string? symbol?)
> >                 #:headers (listof (or/c bytes? string?))
> >                 #:data (or/c false/c bytes? string?))
> >         (values bytes? (listof bytes?) input-port?))]
> >
> > Compared to net/url,
> > - It supports bytes and strings everywhere
> > - It supports data on every method and not just POST
> > - It always returns the status line, headers, and content (as a port)
> >
> > I feel that the only thing it could do better is support two more
> > options for #:data:
> > - A input-port? to read from and copy to the HTTP connection
> > - A (-> output-port? void) function to call with the HTTP connection's
> > output port to stream the data
> >
> > But I'd like a second opinion before adding them.
> >
> > Jay
> >
> > --
> > Jay McCarthy <jay at cs.byu.edu>
> > Assistant Professor / Brigham Young University
> > http://faculty.cs.byu.edu/~jay
> >
> > "The glory of God is Intelligence" - D&C 93
> > _________________________
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
>



-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

"The glory of God is Intelligence" - D&C 93
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.racket-lang.org/dev/archive/attachments/20130904/c3d71f78/attachment.html>