Perl-Critic/PPI

t/21_exhaustive.t: random failures (PPI-1.262)

eserte opened this issue · 5 comments

eserte commented

It seems that random failures in t/21_exhaustive.t are possible. One example:

# input:  "c\${&^!bs%r?;^0wm.1._,q\n*r^]Xg:~|\t<&'_Vs;mr|/\n}=\n-#9!>x9>='_<#,'#^]>9{qmg^%'g( /qytc.?,\n ~f&>0+&gy&V,\@&fg>/f {!+-bVs9?9c%1\"1^{(_r)w`/%~qb>*|#,f,-1]~<c\$`g#19\"\t]s;*=,f_:V`f<;\"%\@\\V0\$^;W~bz;?!]9."
# output: "c\${&^!bs%r?;^0wm.1.,q\n*r^]Xg:~|\t<&'_Vs;mr|/\n}=\n-#9!>x9>='_<#,'#^]>9{qmg^%'g( /qytc.?,\n ~f&>0+&gy&V,\@&fg>/f {!+-bVs9?9c%1\"1^{(_r)w`/%~qb>*|#,f,-1]~<c\$`g#19\"\t]s;*=,f_:V`f<;\"%\@\\V0\$^;W~bz;?!]9."
# shorted failing substring: ""

#   Failed test 'round trip successful'
#   at t/21_exhaustive.t line 112.
# Looks like you failed 1 test of 1114.
t/21_exhaustive.t ................ 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1114 subtests

Another example:

Use of uninitialized value $round_tripped in string eq at t/21_exhaustive.t line 139.
# input:  "_+#8r\tt`-[\n<+0cXV!=.?c!c0=)<<~')19_-b+?WW'[z q-y?#m\@Wt>.'.xg=}|*w|:+[|9}\"mW:Xf()?(cx9*<\t\nVx\$*=z09 z&VX`~1/^\\/,+f}\$_[%q{VW,/b_`9wxx\n\n:;gg\")~\n~x{)f1<'~y1qsf.-b\@V- `;9t\\>{`9-'\$[\@zt;x>%_xt=(\t!\@0"
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 30.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 31.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 32.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 33.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 34.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 35.
Use of uninitialized value $round_tripped_quoted in concatenation (.) or string at t/21_exhaustive.t line 144.
# output: ""
# shorted failing substring: "_+#8r\tt`-[\n<+0cXV!=.?c!c0=)<<~')19_-b+?WW'[z q-y?#m\@Wt>.'.xg=}|*w|:+[|9}\"mW:Xf()?(cx9*<\t\nVx\$*=z09 z&VX`~1/^\\/,+f}\$_[%q{VW,/b_`9wxx\n\n:;gg\")~\n~x{)f1<'~y1qsf.-b\@V- `;9t\\>{`9-'\$[\@zt;x>%_xt=(\t!\@0"

#   Failed test 'round trip successful'
#   at t/21_exhaustive.t line 112.
# Looks like you failed 1 test of 1114.
t/21_exhaustive.t ................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1114 subtests
wchristian commented

Okay, yeah, cool. I broke that. Thanks. :)

eserte commented

It seems that t/21_exhaustive.t can still fail with PPI-1.264. A sample fail report already on CPAN Testers:
http://www.cpantesters.org/cpan/report/bcd8f890-6bd5-11e9-9761-74b8c862b317

And a failure on one of my smokers:

Use of uninitialized value $round_tripped in string eq at t/21_exhaustive.t line 139.
# input:  "<<~` (y0<\"wsxV?[\tV q`+\"<X)[/b&g+.{\\`b/)z]\\%=?s\@ x,`\@1q,^{Vs0)gtm_^0*;>{/*9's,\t!\\*\"r\n%*:>?qzqs|0#m0!-\"'=\t\@^{\$8z&ww[(WX-Vb:[%(y0\${g[ttWzX_*=r_=9==&c|8W\"wq1|%[_\\[/)8<+%x8&s+W+8X:gz?b%q1xft&x&)f"
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 30.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 31.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 32.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 33.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 34.
Use of uninitialized value $quotable in substitution (s///) at t/lib/PPI/Test.pm line 35.
Use of uninitialized value $round_tripped_quoted in concatenation (.) or string at t/21_exhaustive.t line 144.
# output: ""
# shorted failing substring: ""

#   Failed test 'round trip successful'
#   at t/21_exhaustive.t line 112.
# Looks like you failed 1 test of 1114.
t/21_exhaustive.t ................ 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1114 subtests
wchristian commented

All of those appeared to be related to unescaped parens sneaking into regexes. I've improved the tooling around those those tests, fixed the code, and added your examples as regression tests. Released as https://metacpan.org/release/MITHALDU/PPI-1.266

Thanks again. :)