Log/report initial state to help repeat/diagnose generator errors
While running xsmith-cish
(7b989d68) just now (as a server, outside DrRacket), I got a crash. In the terminal window where I was running the server, I saw:
error while unifying types: #(struct:base-type int) and #<type-variable #(struct:base-type bool)>
for node of AST type: GreaterOrEqualExpression
with parent of AST type: VolatileInitializer
Servlet (@ /servlets/standalone.rkt) exception:
unify!: Can't unify types: #(struct:base-type bool) #(struct:base-type int)
context...:
/Applications/Racket/collects/racket/contract/private/arrow-val-first.rkt:390:18
/Users/eeide/git/xsmith/xsmith/private/core-properties.rkt:886:0: xsmith_type-info-func
/Users/eeide/Library/Racket/7.2/pkgs/racr/core.rkt:1183:11
...e-properties.rkt:1024:20
/Users/eeide/Library/Racket/7.2/pkgs/racr/core.rkt:1183:11
/Users/eeide/git/xsmith/xsmith/private/core-properties.rkt:849:0: xsmith_type-constraint-from-parent-func
/Users/eeide/Library/Racket/7.2/pkgs/racr/core.rkt:1183:11
/Users/eeide/git/xsmith/xsmith/private/core-properties.rkt:886:0: xsmith_type-info-func
/Users/eeide/Library/Racket/7.2/pkgs/racr/core.rkt:1183:11
/Users/eeide/git/xsmith/xsmith/private/core-properties.rkt:1070:0: satisfies-type-constraint?
xsmith_apply-choice-filters method in XsmithBaseNodecishChoice%
loop
/Users/eeide/git/xsmith/xsmith/private/grammar-macros.rkt:529:2: xsmith_hole->replacement-function
/Users/eeide/Library/Racket/7.2/pkgs/racr/core.rkt:1183:11
/Users/eeide/git/xsmith/xsmith/private/grammar-macros.rkt:441:9: fill-in
/Applications/Racket/collects/racket/private/list.rkt:115:2: findf
...
This backtrace isn't enough for me to construct a reportable test case, because I don't know the initial conditions of the generator.
If I recall, Csmith worked around this issue by outputting the header comment early, before generating any code. Thus, if the generator crashed, at least its initial state was saved, and the generator could be re-run to hopefully reproduce the bug.
Xsmith could take the same approach or to something else: try to include the state in the backtrack, log the state to a "well known location," etc.