Tests fail when command is not installed.
hartzell opened this issue · 9 comments
I'm going to gamble on the useful of an information-impoverished bug report. I'm not sure I'm going to have time to root out all of the issues, which is lame, but on the other hand at least this might call attention to the issue.
I'm trying to install a copy of the current, f48f0a5f5d, version of bioperl-run. I have very few of the underlying commands on my path. When running tests I get the following failures.
t/BWA.t (Wstat: 512 Tests: 25 Failed: 0)
TODO passed: 24
Non-zero exit status: 2
Parse errors: Bad plan. You planned 36 tests but ran 25.
t/Bowtie.t (Wstat: 512 Tests: 34 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 73 tests but ran 34.
t/Gumby.t (Wstat: 65280 Tests: 8 Failed: 1)
Failed test: 1
Non-zero exit status: 255
Parse errors: Bad plan. You planned 124 tests but ran 8.
t/MAFFT.t (Wstat: 512 Tests: 6 Failed: 2)
Failed tests: 5-6
Non-zero exit status: 2
Parse errors: Bad plan. You planned 23 tests but ran 6.
t/MCS.t (Wstat: 65280 Tests: 4 Failed: 2)
Failed tests: 1-2
Non-zero exit status: 255
Parse errors: Bad plan. You planned 24 tests but ran 4.
t/Maq.t (Wstat: 0 Tests: 51 Failed: 0)
TODO passed: 24
t/Match.t (Wstat: 65280 Tests: 1 Failed: 1)
Failed test: 1
Non-zero exit status: 255
Parse errors: Bad plan. You planned 7 tests but ran 1.
t/PhastCons.t (Wstat: 65280 Tests: 6 Failed: 1)
Failed test: 1
Non-zero exit status: 255
Parse errors: Bad plan. You planned 181 tests but ran 6.
t/SABlastPlus.t (Wstat: 512 Tests: 5 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 69 tests but ran 5.
t/Samtools.t (Wstat: 512 Tests: 24 Failed: 0)
TODO passed: 24
Non-zero exit status: 2
Parse errors: Bad plan. You planned 40 tests but ran 24.
t/StandAloneFasta.t
In the cases I've tracked down so far the root of the problem is that "someone" overrode/reimplemented WrapperBase's executable method but the version that is getting called does not throw an exception if the command is not present. For example, BWA, which has AssemblerBase in it's ISA, which ...., which ends up in WrapperBase::executable, which returns an undef instead of die'ing. That undef foils the -requires_executable test and blooey.
I believe that the others are similar, will dig a bit more before I just pull out my sawzall.
I can replicate the above test failures, and some of the failures are because the wrapper modules are trying to load modules that apparently have been removed from BioPerl. For example, the Match.t test file uses Bio::Tools::Run::Match, which uses Bio::FeatureIO, a module that appears to have been removed from core BioPerl. The same is true for Gumby.t, which uses Bio::Tools::Run::Phylo::Gumby. This module tries to load Bio::SeqFeature::Annotated, which cannot be found since it was removed.
I think the only way to fix this specific issue would be to rework each module and test.
Isn't the issue that these different components are moved into their own package - so I don't entirely know how tests should operate when it comes to not having an expected core anymore.
I think we (maybe I) will have to bite the bullet and explictly specify the dependencies on split-out modules. The tests should/can take this into account and skip rather than fail.
@majensen Yes, that's the general idea.
Re: Bio::FeatureIO, that was moved out of core primarily b/c the implementation was (to be nice) problematic. I am working on an intermediate solution, namely releasing Bio::FeatureIO as a separate dist on CPAN (including Bio::SeqFeature::Annotated)
I have a post on this here if anyone wants a bit of history.
I am doing a major refactoring of the build process in topic/issue11 branch. The requirements/design manifesto is at https://github.com/bioperl/bioperl-run/blob/topic/issue11/maintenance/notes.txt
Making some considerable progress here. It is a complete bitch. I say that with the utmost respect for this distro.
I don't think it's a problem if a wrapper module is installed w/o the requisite executables, as long as it exits cleanly (e.g. if the exec isn't present it dies in a way that can be reproducibly caught).
This would be something preferentially caught early, at object init. @sendu set up some basic infrastructure to handle things at the test level I think, but maybe it just needs to be something more ingrained as a WrapperBase/etc method.
I think the lack of the executable is a secondary problem (and handled fine with sendu's stuff-- in that tests get skipped, rather than failing) to the issue of actual module dependencies. That's just MHO. I just posted at length to the list-- the branch now contains a brave experiment!
Very cool! I'll try it out, post thoughts on list.
Spring cleaning, closing.