[racket] Yet another garbage collection question: major GC cycles

I think the periodic two major collections are coming from the default 
servlet "manager", which defaults to a manager created by 
make-threshold-LRU-manager. (See the two calls to (collect-garbage) in 
web-server/managers/lru.rkt).

Ryan


On 02/15/2013 02:16 PM, Galler wrote:
> Dear Racketeers,
>
> *SMALL PREAMBLE*
>
> I'm trying to understand the observed GC behavior so I can create better
> application performance, and I'm focusing on the major GC-cycle.
>
> For usability, my desire would be to have a major-garbage-collection
> cycle last about the same time as a typical request-response cycle, so
> that users experience a maximum  2x normal delay if a request-response
> cyle is interrupted by a major GC-cycle.
>
> Please feel free to direct me to a paper if its simpler.
>
>
> *THE OBSERVED RACKET BEHAVIOR*
>
> When an application is compiled and run, the garbage collector in v5.3.2
> appears to run through a number of minor collection cycles, then will
> execute two sequential major cycles, in rapid succession.
>
> (I assume this description of behavior is not controversial. For an
> example, please compile and run the source code below.)
>
> The major cycles take a finite amount of time, such duration unrelated
> to the amount of memory cleaned up. This seems consistent with earlier
> descriptions by Matthias that the GC walks through memory looking for
> unreachable code.
>
> The duration of the major cycle seems directly related to the amount of
> memory use by the running application, which again, is consistent with
> prior GC descriptions on this board.
>
> Given the amount of memory usage as an independent vairable, I believe I
> could closely estimate the duration of a major GC cycle (for a specific
> piece of hardware - the constant 3.5ms per 80MB yields good results).
>
> After the major cycles are done, even if no additional user activity
> occurs, the garbage collector will periodically run two sequential major
> cycles, which recover little or no memory.
>
> In contrast, frequency of minor cycles are clearly user-activity dependent.
>
> *THE QUESTIONS*
>
> My questions are
>
> 1) Why are two successive major cycles neccesary?
>
> 2) Why do major cycles occur in the absence of activity? Is there a
> timer associated with major cycles?
>
> 3) Is there an optimization going on between amount of memory
> consumption that triggers the first major cycle and Racket's estimate of
> the RATE of memory consumption? (fast as. frequent vs. slow and infrequent)
>
> 4) is there any rate-related term in the GC algorithm?
>
> 5) How does Racket determine how much memory to ask the operating system
> to allocate to an executable?
>
>
> Thanks
>
> R./
> Zack
>
> EXAMPLE CODE
>
> #lang web-server
> (require  web-server/servlet-env)
>
> (struct gc-info (major? pre-amount pre-admin-amount code-amount
>                          post-amount post-admin-amount
>                          start-process-time end-process-time
>                          start-time end-time)
>    #:prefab)
>
>
> (define log-rcvr (make-log-receiver (current-logger) 'debug 'GC))
>
> (thread (λ _ (letrec ((rfc (λ _ (let ((gc (vector-ref (sync log-rcvr) 2)))
>                                      (printf "\nMajor cycle:? ~A ;
> cleaned-up ~A MB in ~A ms\n"
>                                              (gc-info-major? gc)
>                                              (floor (/ (-
> (gc-info-pre-amount gc) (gc-info-post-amount gc)) (* 1024 1024)))
>                                              (-
> (gc-info-end-process-time gc) (gc-info-start-process-time gc)))
>                                      (rfc)))))
>                   (rfc))))
>
>
> (define (start request)
>    (letrec ((response-generator (λ (make-url)
>                                   (displayln 'clicky)
>                                (response/xexpr `(html (head)
>                                                       (body (a ((href
> ,(format "~A" (make-url receive-request)))) "click me"))))))
>             (receive-request (λ (request)
>                                (response/xexpr `(html (head)
>                                                       (body "Thank you.
> We are done."))))))
>      (send/suspend/dispatch response-generator)))
>
> (serve/servlet start
>                 #:stateless? #t
>                 #:launch-browser? #t
>                 #:connection-close? #t
>                 #:quit? #f
>                 #:listen-ip #f
>                 #:port 8000
>                 #:servlet-path "/")
>
> ____________________
>   Racket Users list:
>   http://lists.racket-lang.org/users