[racket] HTTP-POST byte string gets truncated

The different representations can be inconvenient. Plus sometimes it's
easiest to deal with headers as what they really mean to you: A
dictionary of keys and values. If you do this a lot, you may end up
creating some helpers to convert among representations like the
following.

(define/contract (heads-list->string xs)
  ((listof string?) . -> . string?)
  (string-append (string-join xs "\r\n") "\r\n\r\n"))

(define/contract (heads-string->list s)
  (string? . -> . (listof string?))
  (filter (lambda (s) (not (equal? s "")))
          (regexp-split "\r\n" s)))

(define/contract (heads-list->dict xs)
  ((listof string?) . -> . dict?)
  (for/list ([x (in-list xs)])
    (match x
      [(pregexp "^(\\S+)\\s*:\\s*(.*?)$" (list _ k v)) (cons k v)]
      [else (error 'heads-list->dict "bad header: ~a" x)])))

(define/contract (heads-dict->list d)
  (dict? . -> . (listof string?))
  (for/list ([(k v) (in-dict d)])
    (format "~a: ~a" k v)))

(module+ test
  (require rackunit)
  ;; Round-trip?
  (define header (list "a: 1" "b: 2" "c: 3"))
  (check-equal? (heads-string->list (heads-list->string header)) header)
  (check-equal? (heads-dict->list (heads-list->dict header)) header)
  ;; Edge cases?
  (check-equal? (heads-list->string '()) "\r\n\r\n")
  (check-equal? (heads-string->list "\r\n\r\n") '())
  (check-equal? (heads-list->dict '()) '())
  (check-equal? (heads-dict->list '()) '()))

I just wrote this now. There are some oversimplifications like the
fact that HTTP headers may duplicate keys. But you get the idea.

On Mon, Oct 8, 2012 at 8:04 AM, Mikko Tiihonen
<mikko.tiihonen at tmtiihonen.fi> wrote:
> Thanks, Greg and Danny!
>
> Problem solved - now the POST parameters are also correct! Seems that the
> different representation of headers in net/head caused the POST parameters
> following the header being parsed incorrectly.
>
> Cheers,
> -Mikko
>
>
> On 8.10.2012, at 14:52, Greg Hendershott wrote:
>
> validate-header validates the header as one string, but post-pure-port
> requires that the header is contained in a list of strings ("optional list
> of strings can be used to send header lines to the server"). Is there some
> kind of a mismatch between net/url and net/head?
>
>
> Yes, unfortunately they represent headers differently. net/head
> represents headers as a single "string" or #"bytes string" consisting
> of \r\n separated elements. This is closest to the headers in real
> life.  However net/url uses a list of elements (and they're "string"
> elements only; not #"byte string").
>
> Since you're using post-pure-port from net/url, I think you want:
>
> (define header
>  (list "Host: localhost:8080"
>        "Connection: keep-alive"
>        "User-Agent: Mozilla/5.0 (testClient.rkt 0.1)"
>        "Accept-Encoding: gzip"
>        "Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
>        "Accept-Language: en-us"
>        "Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7"
>        "Cache-control: no-cache"))
> (post-pure-port my-url my-data header)
>
>
> On Mon, Oct 8, 2012 at 3:07 AM, Mikko Tiihonen
> <mikko.tiihonen at tmtiihonen.fi> wrote:
>
> Hi, Danny,
>
>
> the header is here:
>
>
> (define header (list (insert-field #"Host" #"localhost:8080"
>
>                                 (insert-field #"Connection" #"keep-alive"
>
>                                               (insert-field #"User-Agent"
> #"Mozilla/5.0 (testClient.rkt 0.1)"
>
>                                                             (insert-field
> #"Accept-Encoding" #"gzip"
>
>
> (insert-field #"Accept"
> #"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
>
>
> (insert-field #"Accept-Language" #"en-us"
>
>
> (insert-field #"Accept-Charset" #"ISO-8859-1,UTF-8;q=0.7,*;q=0.7"
>
>
> (insert-field #"Cache-control" #"no-cache" empty-header))))))))))
>
>
> For some reason the request struct on the server side seems to get the POST
> data four bytes off... I also noticed that empty-header does not evaluate to
> "\r\n\r\n" as specified in net/head documentation:
>
> empty-header
>
> "\r\n"
>
>
> validate-header validates the header as one string, but post-pure-port
> requires that the header is contained in a list of strings ("optional list
> of strings can be used to send header lines to the server"). Is there some
> kind of a mismatch between net/url and net/head?
>
>
> Cheers,
>
>
> -Mikko
>
>
>
> On 8.10.2012, at 2:49, Danny Yoo wrote:
>
>
> On Sat, Oct 6, 2012 at 4:50 AM, Mikko Tiihonen
>
> <mikko.tiihonen at tmtiihonen.fi> wrote:
>
> Hi, again!
>
>
> I'm continuing to build a small HTTP-client. The problem is now that the
> POST parameter/value byte strings sent by put-pure-port and post-pure-port
> seem to get truncated somewhere. The request-post-data/raw shows that the
> byte string gets prepended with "\r\n\r\n" and truncated by four bytes. E.g.
>
>
> (post-pure-port uri #"param1=hello&param2=world" header)
>
>
> is received as
>
>
> #"\r\n\r\nparam1=hello&param2=w"
>
>
>
> Odd.  What's the content of 'header' here?  It's the only free
>
> variable I see whose value I don't quite understand yet.
>
>
>
> ____________________
>
>  Racket Users list:
>
>  http://lists.racket-lang.org/users
>
>