apjanke/octave-testify

Way too many failures in test suite

Closed this issue · 4 comments

This looks like something wrong with the test runner, not Octave itself.

Summary:

  GNU Octave Version: 5.1.0 (hg id: d05d6eebde10 + patches)
  Tests run on angharad.local (macOS) at 20-Mar-2019 10:08:13
  Test execution time: 69 s

  PASS                            16808
  FAIL                             2015
  REGRESSION                          6
  XFAIL (reported bug)              266
  SKIP (missing feature)             20
  SKIP (run-time condition)          32

  Failed tests:
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/besselj.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/bsxfun.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/cellfun.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/conv2.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/data.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/dlmread.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/ellipj.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/errwarn.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/file-io.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/find.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/gsvd.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/hash.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/inv.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/kron.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/lu.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/mappers.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/max.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/psi.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/qz.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/rand.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/rcond.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/regexp.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/schur.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/sparse-xpow.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/strfns.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/symtab.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/tsearch.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/utils.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/variables.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/dldfcn/amd.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/dldfcn/chol.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/dldfcn/gzip.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/dldfcn/qr.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-cx-diag.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-fcn-handle.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-fcn-inline.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-java.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-lazy-idx.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-struct.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/parse-tree/pt-eval.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/parse-tree/pt-tm-const.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/liboctave/array/Array.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/liboctave/array/Sparse.cc-tst
[...]

Partially fixed in ec208e7 and d752a9c.

Now I'm down to this:

Summary:

  GNU Octave Version: 5.1.0 (hg id: d05d6eebde10 + patches)
  Tests run on angharad.local (macOS) at 19-Apr-2019 15:44:03
  Test execution time: 73 s

  PASS                            18805
  FAIL                               40
  XFAIL (reported bug)              250
  SKIP (missing feature)             20
  SKIP (run-time condition)          32

  Failed tests:
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/bsxfun.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/cellfun.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/conv2.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/data.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/dlmread.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/file-io.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/find.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/gsvd.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/hash.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/mappers.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/max.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/rand.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/sparse-xpow.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/strfns.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/symtab.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/dldfcn/gzip.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-cx-diag.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-java.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/parse-tree/pt-tm-const.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/liboctave/array/Sparse.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/liboctave/util/oct-inttypes.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/elfun/asech.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/bitset.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/interp2.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/logspace.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/num2str.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/postpad.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/prepad.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/quadgk.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/rat.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/trapz.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/geometry/rectint.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/io/importdata.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/legacy/strread.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/miscellaneous/inputname.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/miscellaneous/orderfields.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/optimization/fzero.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/draw/hist.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/draw/isocaps.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/draw/isosurface.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/draw/reducevolume.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/util/copyobj.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/util/hgsave.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/set/setdiff.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/sparse/eigs.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/specfun/betainc.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/specfun/expint.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/specfun/gammainc.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/specfun/perms.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/specfun/realpow.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/statistics/median.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/statistics/quantile.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/statistics/zscore.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/strings/base2dec.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/strings/native2unicode.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/strings/strcat.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/strings/strsplit.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/testfun/test.m

Aw, crap. At least some of the BISTs are depending on the internal implementation details of test.

Here's some test code from inputname.m:

## Warning: heap big magic in the following tests!!!
## The test function builds a private context for each test, with only the
## specified values shared between them.  It does this using the following
## template:
##
##     function [<shared>] = testfn (<shared>)
##        <test>
##     endfunction
##
## To test inputname, I need a function context invoked with known parameter
## names.  So define a couple of shared parameters, et voila!, the test is
## trivial.

%!shared hello, worldly
%!assert (inputname (1), "hello")
%!assert (inputname (2), "worldly")
%!assert (inputname (3), "")

## Clear parameter list so that testfn is created with zero inputs/outputs
%!shared
%!assert (inputname (-1), "")
%!assert (inputname (1), "")

That seems to rely on the fact that Octave's existing test implements shared variables by passing them as parameters to a testing function call that it generates dynamically, and that that testing function (with shared variables passed as named parameters) is the immediately enclosing stack frame in which statements in the test block are executed. That's incompatible with the object-managed workspace approach that Testify is using.

Fixed spurious inclusion of files with XFAILs in the files-with-failed-tests list in 480e151.

Now I've got this:

Summary:

  GNU Octave Version: 5.1.0 (hg id: d05d6eebde10 + patches)
  Tests run on angharad.local (macOS) at 19-Apr-2019 16:33:25
  Test execution time: 679 s

  PASS                            18807
  FAIL                               38
  XFAIL (reported bug)              250
  SKIP (missing feature)             20
  SKIP (run-time condition)          32

  Failed tests:
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/corefcn/cellfun.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/etc/tests/libinterp/octave-value/ov-java.cc-tst
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/general/logspace.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/miscellaneous/inputname.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/draw/isocaps.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/draw/isosurface.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/draw/reducevolume.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/util/copyobj.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/plot/util/hgsave.m
     /Applications/Octave-5.1.0.app/Contents/Resources/usr/Cellar/octave-octave-app@5.1.0/5.1.0/share/octave/5.1.0/m/sparse/eigs.m

That took care of the big issues that were causing tons of failures. Calling this fixed. We'll grind down the remaining failures under other issues.

Followed up by #72.