[racket] net/http-client

On Tue, Sep 24, 2013 at 12:31 PM, Evan Donahue <emdonahu at gmail.com> wrote:
> Hmm, I'm no expert on http cookies, but here are the sticking points (or
> possibly misinterpretations of the spec and/or the library) that have been
> holding me up (referencing rfc2965):
>
> 1) the print-cookie function generates path, domain, comment, etc fields
> that don't belong in a Cookie header, so to use print-cookie the cookie must
> only contain name and value (and no alternative print can be written due to
> lack of accessors).
>
> 2) Since the domain/path etc cannot be accessed, this information would have
> to be stored in whatever container was holding your set of cookies for
> selecting the appropriate cookies to send (an interface you note should
> probably be better supported).
>
> 3) Even with accessors, the domain mutator validates the domain as requiring
> a leading '.' which, if no domain is specified, I believe bars the cookies
> from holding the default value of the fully qualified host.
>
> Also, one question: what do you mean by "There's no plans to add this, but
> it should be very easy to anyone that has a use to expose the right stuff
> from net/cookie." ?

I mean

1. I have no plans to change net/cookie.

2. net/cookie is really simple, so if you feel like you know anything
about cookies, you should be able to change it and make it do what you
want. (Or actually, I'd be surprised if it does anything well enough
that you'd want to keep any of it when making a new interface that
works better for the client, if you felt like the client interface
were not good.)

Jay

> Thanks,
> Evan
>
>
> On Tue, Sep 24, 2013 at 5:33 AM, Jay McCarthy <jay.mccarthy at gmail.com>
> wrote:
>>
>> I believe the net/cookie was intended to be used for both client and
>> server. For instance, print-cookie turns a cookie structure into a
>> string, which is a header that can be used on either side. I think the
>> only thing that would be needed as far as structure accessors would be
>> a way to find the cookies for a given path from a collection of
>> cookies. There's no plans to add this, but it should be very easy to
>> anyone that has a use to expose the right stuff from net/cookie.
>>
>> I do think net/cookie is ripe to be improved though. I think something
>> more like web-server/http/cookie would be better than the interface it
>> provides.
>>
>> Jay
>>
>>
>> On Mon, Sep 23, 2013 at 8:21 PM, Evan Donahue <emdonahu at gmail.com> wrote:
>> > As long as people are discussing http client libraries, I've been
>> > wondering
>> > what the status of net/cookie is. It seems that it's geared towards
>> > server-side cookie management (and I don't see an obvious way to
>> > generate
>> > Cookie: headers from net/cookie structs since none of the accessors are
>> > exported). Is there an existing or planned cannonical solution to
>> > managing
>> > cookies on the client side?
>> >
>> > Thanks,
>> > Evan
>> >
>> >
>> > On Thu, Sep 19, 2013 at 11:21 AM, Erik Pearson <erik at adaptations.com>
>> > wrote:
>> >>
>> >> Just finishing up a two day wrestling match with net/url, couchdb, and
>> >> assorted other incomplete libraries ... I have some fresh insight.
>> >>
>> >> First, the new http client library will be very welcome!
>> >>
>> >> A quick review of the existing code relieves me of most of my worries
>> >> about net/url, which is a bit of a beautiful mess. In particular, I
>> >> think
>> >> that some of the code that analyzes the header is too brittle. In
>> >> net/url a
>> >> header is expected to be nearly perfectly formed. For instance, the
>> >> "chunked" condition is tested for by a match against the literal header
>> >> string "Transfer-Encoding: chunked". Just a difference in case, or an
>> >> extra
>> >> space somewhere will lead to a false negative. In the new code, this is
>> >> done
>> >> by a much more forgiving regexp comparison. However, I'm not sure this
>> >> goes
>> >> far enough. More on that in a sec.
>> >>
>> >> I would like to make some recommendations:
>> >> - keep text as bytstrings -- it looks like this is the way the new
>> >> library
>> >> works (but not net/url.)
>> >> - the status line should be parsed and the status code converted to an
>> >> integer.
>> >> - The header should be parsed into the simplest usable form. I would
>> >> suggest that an alist, with the key being a lower cased symbol and the
>> >> value
>> >> being the original bytestring.
>> >> - When forming a header for a request, the same format should be used.
>> >> I
>> >> realize that if one wants proper-cased field names, the library will
>> >> need to
>> >> perform this formatting.
>> >> - All information should be made available back to the api user.
>> >> - Standard header field values should be optionally parsed
>> >>
>> >> These changes would make the library much more useful. It would be
>> >> consistent with http library design across languages. It would also
>> >> promote
>> >> more uniformity across Racket libraries which use them. I've found that
>> >> when
>> >> diving into a library (currently I'm dealing with couchdb) that has to
>> >> reinvent the http wheel -- it may be reinvented with a slight twist. If
>> >> one
>> >> is using multiple libraries together, each with a different idea of
>> >> what a
>> >> header or https status line is, well, things get really unnecessarily
>> >> complicated.
>> >>
>> >> I think the core library could benefit from this as well. For instance,
>> >> when looking for fields in the header, it would be more reliable to
>> >> find the
>> >> field in an alist, than to use an regex to extract values out of the
>> >> bytestring. This is a very common operation when dealing with http, and
>> >> I
>> >> think it should be as simple and reliable as practical.
>> >>
>> >> Finally I would recommend that there be a facility for decoding and
>> >> encoding standard header fields. The application of the parsing would
>> >> be
>> >> optional, but I think it is important to have an implementation in the
>> >> core
>> >> library. The header field names and formats are well specified, and
>> >> there
>> >> are not all that many of them. They could be supplied as a hash or
>> >> aref,
>> >> with the field name lower-cased symbol (as in the parsed header) as key
>> >> and
>> >> two functions, one to decode from a bytstring, and one to encode into a
>> >> bytestring. The simplest, most racketiest data structure would be best.
>> >> A
>> >> library user could grab this translation database and extend or modify
>> >> it at
>> >> will, but they would have a great head start.
>> >>
>> >> Anyway, this is far from exhaustive, but these are some of the issues
>> >> I've
>> >> grappled with in recent hours and are right on the top of my head.
>> >>
>> >>
>> >> On Tue, Sep 17, 2013 at 6:51 AM, Jay McCarthy <jay.mccarthy at gmail.com>
>> >> wrote:
>> >>>
>> >>> I think it's an obvious request, but a character flaw of mine is not
>> >>> doing things unless they can be done really good. In this case, I see
>> >>> a hash table as a "parse" of the headers. It's not obvious to me how
>> >>> to parse them. For example...
>> >>>
>> >>> - The same header can appear many times, so (Key -> Value) is
>> >>> incorrect, unless you overwrite one. It would be better to have (Key
>> >>> -> (Listof Value)) but that feels really ugly since most of the time
>> >>> there will just be one
>> >>> - The spec doesn't mandate case sensitivity on headers, so I would
>> >>> need to canonicalize "ACCept-ENCodiNG" to something else. Maybe
>> >>> 'Accept-Encoding?
>> >>> - The value of many headers is not really a string. For instance,
>> >>> Content-Length is a number, Cache-Control is an association list,
>> >>> Content-Disposition is complicated, etc. I feel like it is
>> >>> disingenuous to only partial parse.
>> >>> - Dealing with all this may be wasted effort for most requests that
>> >>> just care about the body
>> >>>
>> >>> For these reasons, I think http-client should just return the list of
>> >>> bytes. I think it would be nice to have another function that parses
>> >>> that so clients could optionally call it if it is important. That can
>> >>> be part of http-client.
>> >>>
>> >>> Jay
>> >>>
>> >>>
>> >>>
>> >>> On Tue, Sep 17, 2013 at 3:41 AM, adam moore <nerdfunk at gmail.com>
>> >>> wrote:
>> >>> > Hi Jay,
>> >>> >
>> >>> > Just looking over the new http client code - looking very nice, and
>> >>> > much better than my current slapped together parsing of
>> >>> > get-impure-port.
>> >>> >
>> >>> > I was wondering if it might be better to pass back the headers as
>> >>> > something easier to look up against, for example as a hash table? Or
>> >>> > perhaps, provide an option to do so. I think it's a pretty common
>> >>> > use
>> >>> > case to provide for.
>> >>> >
>> >>> > Thanks again,
>> >>> > Adam
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Jay McCarthy <jay at cs.byu.edu>
>> >>> Assistant Professor / Brigham Young University
>> >>> http://faculty.cs.byu.edu/~jay
>> >>>
>> >>> "The glory of God is Intelligence" - D&C 93
>> >>> ____________________
>> >>>   Racket Users list:
>> >>>   http://lists.racket-lang.org/users
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Erik Pearson
>> >> Adaptations
>> >> ;; web form and function
>> >>
>> >> ____________________
>> >>   Racket Users list:
>> >>   http://lists.racket-lang.org/users
>> >>
>> >
>>
>>
>>
>> --
>> Jay McCarthy <jay at cs.byu.edu>
>> Assistant Professor / Brigham Young University
>> http://faculty.cs.byu.edu/~jay
>>
>> "The glory of God is Intelligence" - D&C 93
>
>



-- 
Jay McCarthy <jay at cs.byu.edu>
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

"The glory of God is Intelligence" - D&C 93