Total failure on `compiler-test`

[samth]

• Check out and install racket/compiler
• raco cover compiler/compiler-test
• lots and lots of errors.

[florence]

Okay I see three types of errors here:

1. "write: cannot marshal value that is embedded in compiled code
   value: #procedure:...cover/strace.rkt:14:2"
2. "require: PLaneT could not find the package "racket-tester/p1": Server had no matching package: No package matched the specified criteria
   in: (submod (planet racket-tester/p1/has-sub) the-sub)"
3. "with-output-to-file: file exists
   path: /var/folders/z9/cyxcw7n11cs8zxqf29pghzy80000gn/T/stdout"
4. "read-language: expected (after whitespace and comments) #lang ' or#!' followed immediately by a language name, found #r"
5. "error: dont-run"

Do these match the errors youre seeing?

1. Cover uses stacktrace@ internally to inject its code coverage instructions, and it looks like this is floating up from there. I will do more digging to see what is going on.
2. We never considered/tested planet packages, so I'm not surprised this breaks. This is worthy of a new issue.
3. This might have something to do with cover grabbing stdout...? I'm not sure. Looking at "/Users/florence/playground/compiler/compiler-test/tests/compiler/embed/embed-planet-2/main.ss" (one of the files that failed) I'm confused. This will require a little more digging...
4. it looks like "read-language" doesn't do #reader forms. Making new issue.
5. Probably a file that raco test was instructed to ignore. Duplicate with Issue #18

[samth]

I'm pretty sure that 1 is also a duplicate of issue #18.

[florence]

With the recent updates to cover we now have a new set of errors:

1. If this the compiler is tested via raco cover -c compiler most errors are replaced by "link: bad variable linkage"
2. "write: cannot marshal value that is embedded in compiled code" still exists. I suspect this is strace interacting poorly
3. "raco: expects [] ... on the command line, given 0 arguments" now happens. I have no clue why it does not in a raco test as the info.rkt does not have test-command-line-arguments and no config submodule is present. This does not appear to effect results. (but its hard to know until 6 is fixed)
4. "error: dont-run
   context...:
   /Users/florence/playground/compiler/compiler-test/tests/compiler/test/d/c.rkt: [running body]"
   This is because cover always loads the parent module. I'm not sure how to tell if a submodule exists without doing so. Logged as #25
5. Compiling "compiler/compiler-test/tests/compiler/zo-test.rkt" takes an extremely long time.
6. "read-language: expected (after whitespace and comments) #lang ' or#!' followed immediately by a language name, found #r" still occurs. previously logged as #24

[florence]

1. I'm fairly sure 1 is a result of a botched reinstall of compiler, given that my stacktrace shows the install in extra-pkgs and the one in collects. @samth can you replicate the ""link: bad variable linkage"" with a raco cover -c compiler
2. Still have no clue what is going on with this.
3. Ditto, but it doesn't seem to effect anything.

Everything else is fixed.

[florence]

There is one issue left here: "write: cannot marshal value that is embedded in compiled code"

this is caused by how stacktrace annotates a program. stacktrace is inherently 3D, which means that attempting to write "zo" file for something compiled by cover (or rather, by strace itself) will fail. I do not know how to fix this, and it may very well be unfixable, sans a complete rewrite of stacktrace.

This problem is specific to programs that force a zo file to be compiled and written while within cover.

[samth]

@mflatt, any thoughts on how stacktrace should handle this?

[mflatt]

I agree that the coverage implementation in stacktrace would have to be changed. It's not that stacktrace overall is inherently 3D, since it's used by errortrace, which successfully eschews 3D annotation. But the profiling and coverage implementation in stacktrace is always 3D, if I remember correctly. It may be a matter of replacing inline boxes with variables that are defined to box calls at the top of a module.

[florence]

@mflatt if I understand you correctly, the way to fix this would be to change:

(define (test-covered key)
      (hash-set! (test-coverage-info) key #t))

to

(define (test-covered stx)
  (define v (hash-ref coverage stx #f))
  (and v
       (with-syntax ([v v])
         #'(#%plain-app set-box! v #t))))

such that instead of having a box be returned from the coverage hashmap, a syntax object is returned from the coverage hashmap, and that object is a #%variable-reference to a box containing the coverage information for that expression. But how would I be sure that that variable-reference would be in scope in that module? i can't just inject definitions into that module because that module may be written in some strange language that doesn't have #%require or define-values. Or do I misunderstand you?

[mflatt]

Yes, something like that. You can inject definitions into the module; stacktrace works on expanded modules, and it can add a define-values form (where the define-values identifier has the right binding) just like anything else that stacktrace adds to the module.

[florence]

With the exception of #28, which requires a user side fix 1639151 closes this issue. :)