<div dir="ltr">This change seemed to change the format of .dep files, likely as intended to add the indirect dependencies. Is there any documentation of what the format is supposed to be? Currently I've just been trying to read cm.rkt and understand how it treats them.</div><div class="gmail_extra"> <div class="gmail_quote">On Thu, Jan 8, 2015 at 9:31 AM, <<a href="mailto:mflatt@racket-lang.org" target="_blank">mflatt@racket-lang.org</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">mflatt has updated `master' from c56c9250f1 to 95e85ec5bd. <a href="http://git.racket-lang.org/plt/c56c9250f1..95e85ec5bd" target="_blank">http://git.racket-lang.org/plt/c56c9250f1..95e85ec5bd</a> =====[ 2 Commits ]====================================================== Directory summary: 45.1% pkgs/racket-doc/scribblings/raco/ 4.7% pkgs/racket-doc/scribblings/reference/ 47.5% racket/collects/compiler/ ~~~~~~~~~~ fe9a04d Matthew Flatt <<a href="mailto:mflatt@racket-lang.org">mflatt@racket-lang.org</a>> 2015-01-08 09:11 : | doc tweaks for `raco {setup,make}` : M pkgs/racket-doc/scribblings/raco/make.scrbl | 4 ++-- M pkgs/racket-doc/scribblings/raco/setup.scrbl | 22 ++++++++++++---------- ~~~~~~~~~~ 95e85ec Matthew Flatt <<a href="mailto:mflatt@racket-lang.org">mflatt@racket-lang.org</a>> 2015-01-08 09:57 : | add support for indirect CM dependencies; use in `lazy-require` | | If module M in package P imports module N from package Q, | and if N has a `lazy-require` for a module in R that is | triggered during the compilation of M, then P doesn't really | depend on R; P depends on Q, and Q depends on R, and P | shoudn't necessarily know anything about Q. At the same time, | a change to the file in R means that M must be recompiled. | So, continue to track the compilation dependency, but mark | it as "indirect" so that the package-dependency checker can | ignore the dependency. : M pkgs/racket-doc/scribblings/raco/make.scrbl | 33 ++++++++----- M racket/collects/compiler/cm-accomplice.rkt | 14 +++--- M racket/collects/compiler/cm.rkt | 49 ++++++++++++++------ M racket/collects/racket/lazy-require.rkt | 2 +- M racket/collects/setup/private/pkg-deps.rkt | 1 + M .../racket-doc/scribblings/reference/syntax.scrbl | 8 ++-- =====[ Overall Diff ]=================================================== pkgs/racket-doc/scribblings/raco/make.scrbl ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- OLD/pkgs/racket-doc/scribblings/raco/make.scrbl +++ NEW/pkgs/racket-doc/scribblings/raco/make.scrbl @@ -123,7 +123,7 @@ would create only @filepath{compiled/b_rkt.zo} and @; ---------------------------------------------------------------------- -@section{Dependency Files} +@section[#:tag "Dependency Files"]{Dependency Files} In addition to a bytecode file, @exec{raco make} creates a file @filepath{compiled/@nonterm{name}_@nonterm{ext}.dep} that records @@ -538,7 +538,7 @@ messages are instances of a @racket[parallel-compile-event] prefab structure: @racketblock[ (struct parallel-compile-event (worker event) #:prefab) -]. +] The worker field is the index of the worker that the created the event. The event field is a @racket[compile-event] as document in @@ -550,25 +550,36 @@ field is a @racket[compile-event] as document in @defmodule[compiler/cm-accomplice] -@defproc[(register-external-file [file (and path? complete-path?)]) void?]{ +@defproc[(register-external-file [file (and path? complete-path?)] + [#:indirect? indirect? any/c #f]) + void?]{ -Logs a message (see @racket[log-message]) at level @racket['info] to -a logger named @racket['cm-accomplice]. The -message data is a @racketidfont{file-dependency} prefab structure type -with two fields; the first field's value is @racket[file] and the second -field's value is @racket[#f] (to indicate a non-module dependency). +Logs a message (see @racket[log-message]) at level @racket['info] to a +logger named @racket['cm-accomplice]. The message data is a +@racketidfont{file-dependency} prefab structure type with two fields; +the first field's value is @racket[file] and the second field's value +is @racket[#f] (to indicate a non-module dependency). If the +@racket[indirect?] argument is true, the data is more specifically an +instance of a @racketidfont{file-dependency/indirect} prefab structure +type that is a subtype of @racketidfont{file-dependency} with no new +fields. A compilation manager implemented by @racketmodname[compiler/cm] looks -for such messages to register an external dependency. The compilation -manager records (in a @filepath{.dep} file) the path as contributing -to the implementation of the module currently being +for such messages to register an external dependency. In response, the +compilation manager records (in a @filepath{.dep} file) the path as +contributing to the implementation of the module currently being compiled. Afterward, if the registered file is modified, the -compilation manager will know to recompile the module. +compilation manager will know to recompile the module. An ``indirect'' +dependency has no effect on recompilation, but it can signal to other +tools, such as a package-dependency checker, that the dependency is +indirect (and should not imply a direct package dependency). The @racket[include] macro, for example, calls this procedure with the path of an included file as it expands an @racket[include] form.} -@defproc[(register-external-module [file (and path? complete-path?)]) void?]{ +@defproc[(register-external-module [file (and path? complete-path?)] + [#:indirect? indirect? any/c #f]) + void?]{ Like @racket[register-external-file], but logs a message with a @racketidfont{file-dependency} prefab structure type whose second pkgs/racket-doc/scribblings/raco/setup.scrbl ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- OLD/pkgs/racket-doc/scribblings/raco/setup.scrbl +++ NEW/pkgs/racket-doc/scribblings/raco/setup.scrbl @@ -744,13 +744,14 @@ Optional @filepath{info.rkt} fields trigger additional actions by module. More specifically, used modules are determined when deleting a @filepath{.dep} file, which would have been created to accompany a @filepath{.zo} file when the @filepath{.zo} was built - by @exec{raco setup}. If the @filepath{.dep} file indicates another - module, that module's @filepath{.zo} is deleted only if it also has - an accompanying @filepath{.dep} file. In that case, the - @filepath{.dep} file is deleted, and additional used modules are - deleted based on the used module's @filepath{.dep} file, etc. - Supplying a specific list of collections to @exec{raco setup} disables - this dependency-based deletion of compiled files.} + by @exec{raco setup} or @exec{raco make} (see + @secref["Dependency\x20Files"]). If the @filepath{.dep} file + indicates another module, that module's @filepath{.zo} is deleted + only if it also has an accompanying @filepath{.dep} file. In that + case, the @filepath{.dep} file is deleted, and additional used + modules are deleted based on the used module's @filepath{.dep} + file, etc. Supplying a specific list of collections to @exec{raco + setup} disables this dependency-based deletion of compiled files.} ] @@ -816,9 +817,10 @@ with fewer dependencies. @subsection{How Dependency Checking Works} Dependency checking uses @filepath{.zo} files, associated -@filepath{.dep} files, and the documentation index. Dynamic -references, such as through @racket[dynamic-require], are not visible -to the dependency checker; only dependencies via @racket[require], +@filepath{.dep} files (see @secref["Dependency Files"]), and the +documentation index. Dynamic references, such as through +@racket[dynamic-require], are not visible to the dependency checker; +only dependencies via @racket[require], @racket[define-runtime-module-path-index], and other forms that cooperate with @racket[raco make] are visible for dependency checking. pkgs/racket-doc/scribblings/reference/syntax.scrbl ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- OLD/pkgs/racket-doc/scribblings/reference/syntax.scrbl +++ NEW/pkgs/racket-doc/scribblings/reference/syntax.scrbl @@ -3005,10 +3005,10 @@ submodule). Introduced submodules have the names @racket[lazy-require-]@racket[_n]@racketidfont{-}@racket[_m], where @racket[_n] is a phase-level number and @racket[_m] is a number. -When the use of a lazily-required function triggers module loading, -@racket[register-external-module] declares a potential compilation -dependency (in case the function is used in the process of compiling a -module). +When the use of a lazily-required function triggers module loading, it +also triggers a use of @racket[register-external-module] to declare an +indirect compilation dependency (in case the function is used in the +process of compiling a module). @examples[#:eval lazy-require-eval (lazy-require racket/collects/compiler/cm-accomplice.rkt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- OLD/racket/collects/compiler/cm-accomplice.rkt +++ NEW/racket/collects/compiler/cm-accomplice.rkt @@ -3,16 +3,18 @@ (provide register-external-file register-external-module) -(define (register-external-file f) - (register-external 'register-external-file f #f)) -(define (register-external-module f) - (register-external 'register-external-module f #t)) +(define (register-external-file f #:indirect? [indirect? #f]) + (register-external 'register-external-file f #f indirect?)) +(define (register-external-module f #:indirect? [indirect? #f]) + (register-external 'register-external-module f #t indirect?)) -(define (register-external who f module?) +(define (register-external who f module? indirect?) (unless (and (path? f) (complete-path? f)) (raise-type-error who "complete path" f)) (log-message (current-logger) 'info 'cm-accomplice (format "file dependency: ~s" f) - `#s(file-dependency ,f ,module?))) + (if indirect? + `#s((file-dependency/indirect file-dependency 2) ,f ,module?) + `#s(file-dependency ,f ,module?)))) racket/collects/compiler/cm.rkt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- OLD/racket/collects/compiler/cm.rkt +++ NEW/racket/collects/compiler/cm.rkt @@ -231,9 +231,13 @@ (define (get-dep-sha1s deps up-to-date collection-cache read-src-syntax mode roots must-exist? seen) (let ([l (for/fold ([l null]) ([dep (in-list deps)]) (and l + ;; (cons 'indirect dep) => indirect dependency (for pkg-dep checking) ;; (cons 'ext rel-path) => a non-module file, check source ;; rel-path => a module file name, check cache - (let* ([ext? (and (pair? dep) (eq? 'ext (car dep)))] + (let* ([dep (if (and (pair? dep) (eq? 'indirect (car dep))) + (cdr dep) + dep)] + [ext? (and (pair? dep) (eq? 'ext (car dep)))] [p (collects-relative*->path (if ext? (cdr dep) dep) collection-cache)]) (cond [ext? (let ([v (get-source-sha1 p)]) @@ -273,19 +277,26 @@ external-module-deps ; can create cycles if misused! reader-deps))] [external-deps (remove-duplicates external-deps)]) + (define (path*->collects-relative/maybe-indirect dep) + (if (and (pair? dep) (eq? 'indirect (car dep))) + (cons 'indirect (path*->collects-relative (cdr dep))) + (path*->collects-relative dep))) (with-compile-output dep-path (lambda (op tmp-path) (let ([deps (append - (map path*->collects-relative deps) + (map path*->collects-relative/maybe-indirect deps) (map (lambda (x) - (cons 'ext (path*->collects-relative x))) + (define d (path*->collects-relative/maybe-indirect x)) + (if (and (pair? d) (eq? 'indirect d)) + (cons 'indirect (cons 'ext (cdr d))) + (cons 'ext d))) external-deps))]) - (write (list* (version) - (cons (or src-sha1 (get-source-sha1 path)) - (get-dep-sha1s deps up-to-date collection-cache read-src-syntax mode roots #t #hash())) - deps) - op) - (newline op)))))) + (write (list* (version) + (cons (or src-sha1 (get-source-sha1 path)) + (get-dep-sha1s deps up-to-date collection-cache read-src-syntax mode roots #t #hash())) + deps) + op) + (newline op)))))) (define (format-time sec) (let ([d (seconds->date sec)]) @@ -311,6 +322,7 @@ (define-struct ext-reader-guard (proc top) #:property prop:procedure (struct-field-index proc)) (define-struct file-dependency (path module?) #:prefab) +(define-struct (file-dependency/indirect file-dependency) () #:prefab) (define (compile-zo* mode roots path src-sha1 read-src-syntax zo-name up-to-date collection-cache) ;; The `path' argument has been converted to .rkt or .ss form, @@ -322,10 +334,14 @@ (define reader-deps null) (define deps-sema (make-semaphore 1)) (define done-key (gensym)) - (define (external-dep! p module?) + (define (external-dep! p module? indirect?) + (define bstr (path->bytes p)) + (define dep (if indirect? + (cons 'indirect bstr) + bstr)) (if module? - (set! external-module-deps (cons (path->bytes p) external-module-deps)) - (set! external-deps (cons (path->bytes p) external-deps)))) + (set! external-module-deps (cons dep external-module-deps)) + (set! external-deps (cons dep external-deps)))) (define (reader-dep! p) (call-with-semaphore deps-sema @@ -386,7 +402,8 @@ (file-dependency? (vector-ref l 2)) (path? (file-dependency-path (vector-ref l 2)))) (external-dep! (file-dependency-path (vector-ref l 2)) - (file-dependency-module? (vector-ref l 2)))) + (file-dependency-module? (vector-ref l 2)) + (file-dependency/indirect? (vector-ref l 2)))) (loop)))) ;; Write the code and dependencies: @@ -627,9 +644,13 @@ ;; If `sha1-only?', then `maybe-compile-zo' returns a #f or thunk: (maybe-compile-zo sha1-only? deps mode roots path orig-path read-src-syntax up-to-date collection-cache new-seen)] [(ormap - (lambda (p) + (lambda (raw-p) + ;; (cons 'indirect dep) => indirect dependency (for pkg-dep checking) ;; (cons 'ext rel-path) => a non-module file (check date) ;; rel-path => a module file name (check transitive dates) + (define p (if (and (pair? raw-p) (eq? 'indirect (car raw-p))) + (cdr raw-p) + raw-p)) (define ext? (and (pair? p) (eq? 'ext (car p)))) (define d (collects-relative*->path (if ext? (cdr p) p) collection-cache)) (define t racket/collects/racket/lazy-require.rkt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- OLD/racket/collects/racket/lazy-require.rkt +++ NEW/racket/collects/racket/lazy-require.rkt @@ -110,4 +110,4 @@ modpath (variable-reference->resolved-module-path vr))))]) (when (path? path) - (register-external-module path)))) + (register-external-module path #:indirect? #t)))) racket/collects/setup/private/pkg-deps.rkt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --- OLD/racket/collects/setup/private/pkg-deps.rkt +++ NEW/racket/collects/setup/private/pkg-deps.rkt @@ -489,6 +489,7 @@ ;; Treat everything in ".dep" as 'build mode... (define deps (cddr (call-with-input-file* (build-path dir f) read))) (for ([dep (in-list deps)]) + ;; Note: indirect dependencies (which start with 'indirect) are ignored (when (and (pair? dep) (eq? 'collects (car dep))) (define path-strs (map bytes->string/utf-8 (cdr dep))) </blockquote></div> </div>