Do doctests test suites always need to depend on the library?
RyanGlScott opened this issue · 15 comments
RyanGlScott commented
It seems that doctests test suites might always have an implicit dependency on the library being tested. I noticed this when doing the following:
$ cabal get parsers-0.12.5
Unpacking to parsers-0.12.5/
$ cd parsers-0.12.5/
$ cabal configure --enable-tests
Resolving dependencies...
[1 of 1] Compiling Main ( dist/setup/setup.hs, dist/setup/Main.o )
Linking ./dist/setup/setup ...
Configuring parsers-0.12.5...
$ cabal test doctests
Preprocessing test suite 'doctests' for parsers-0.12.5..
Building test suite 'doctests' for parsers-0.12.5..
<no location info>: warning: [-Wmissing-home-modules]
These modules are needed for compilation but not listed in your .cabal file's other-modules: Build_doctests
[1 of 2] Compiling Build_doctests ( dist/build/doctests/autogen/Build_doctests.hs, dist/build/doctests/doctests-tmp/Build_doctests.o )
[2 of 2] Compiling Main ( tests/doctests.hs, dist/build/doctests/doctests-tmp/Main.o )
<no location info>: warning: [-Wmissing-home-modules]
These modules are needed for compilation but not listed in your .cabal file's other-modules: Build_doctests
Linking dist/build/doctests/doctests ...
Running 1 test suites...
Test suite doctests: RUNNING...
<command-line>:7:0: error:
fatal error: dist/build/autogen/cabal_macros.h: No such file or directory
compilation terminated.
-i
-i/home/rgscott/Documents/Hacking/Haskell/parsers-0.12.5/dist/build/autogen
-i/home/rgscott/Documents/Hacking/Haskell/parsers-0.12.5/dist/build
-i/home/rgscott/Documents/Hacking/Haskell/parsers-0.12.5/src
-hide-all-packages
-package-db=dist/package.conf.inplace
-optP-include
-optPdist/build/autogen/cabal_macros.h
-package-id=base-4.10.0.0
-package-id=base-orphans-0.6-9iPDLrUqQ5yHb9fb9OUZ9L
-package-id=charset-0.3.7.1-GkbAILH2Vj9HXMMGXPSeNg
-package-id=containers-0.5.10.2
-package-id=parsec-3.1.11-DPgnR92AWEaFOaixmwipet
-package-id=attoparsec-0.13.1.0-2xtipSJ3oFDBfLwedqccX
-package-id=text-1.2.2.2-EGUst8sqNAZCw1xLPcmcMH
-package-id=transformers-0.5.2.0
-package-id=mtl-2.2.1-19EL8AGBsN3DnnOhrC9xY3
-package-id=scientific-0.3.5.1-L0qYzdp0wpz8rO8gircNSR
-package-id=unordered-containers-0.2.8.0-HVcKYx0GQoVIIbKuoCcJbx
-package-id=bytestring-0.10.8.2
-package-id=directory-1.3.0.2
-package-id=doctest-0.12.0-JcuLvnckae744xvqjXdMWF
-package-id=filepath-1.4.1.2
-package-id=QuickCheck-2.10.0.1-DTIBC3CyU6p3h4xSnOjkg1
-package-id=quickcheck-instances-0.3.16-A15GUudjPiB6fV2AyfxwM5
Text.Parser.Char
Text.Parser.Combinators
Text.Parser.LookAhead
Text.Parser.Permutation
Text.Parser.Expression
Text.Parser.Token
Text.Parser.Token.Style
Text.Parser.Token.Highlight
doctests: `gcc' failed in phase `C pre-processor'. (Exit code: 1)
Test suite doctests: FAIL
Test suite logged to: dist/test/parsers-0.12.5-doctests.log
0 of 1 test suites (0 of 1 test cases) passed.
The culprit appears to be that the doctests test suite in parsers lacks an explicit dependency on the parsers library. If I add that, then cabal test doctests works as expected.
Currently, we advise users in the documentation to depend on the library in a doctests test suite only in the event that one uses cbits, but perhaps we should always be advising this?
phadej commented
@RyanGlScott can you paste the output of cabal test doctest (the test.log) in successful case?
And also find dist ?
phadej commented
Note: there is no need to depend on the library, as we should pass everything to doctests to load the package in GHCi. In that scenari we seem to forget something...
(cbits is different...)
RyanGlScott commented
@RyanGlScott can you paste the output of
cabal test doctest(the test.log) in successful case?
Here's the full output of cabal test doctests:
$ cabal test doctests
[1 of 1] Compiling Main ( dist/setup/setup.hs, dist/setup/Main.o ) [flags changed]
Linking ./dist/setup/setup ...
Preprocessing library for parsers-0.12.5..
Building library for parsers-0.12.5..
[1 of 8] Compiling Text.Parser.Combinators ( src/Text/Parser/Combinators.hs, dist/build/Text/Parser/Combinators.o )
[2 of 8] Compiling Text.Parser.Char ( src/Text/Parser/Char.hs, dist/build/Text/Parser/Char.o )
[3 of 8] Compiling Text.Parser.Expression ( src/Text/Parser/Expression.hs, dist/build/Text/Parser/Expression.o )
src/Text/Parser/Expression.hs:115:15: warning: [-Wmonomorphism-restriction]
• The Monomorphism Restriction applies to the bindings for ‘rassoc’,
‘lassoc’, ‘nassoc’, ‘prefix’,
‘postfix’
Consider giving a type signature for these binders
• In the expression:
let
rassoc, lassoc, nassoc :: [m (a -> a -> a)]
prefix, postfix :: [m (a -> a)]
(rassoc, lassoc, nassoc, prefix, postfix)
= foldr splitOp ([], [], [], [], []) ops
....
in
termP <**> (rassocP <|> lassocP <|> nassocP <|> pure id)
<?> "operator"
In an equation for ‘makeParser’:
makeParser term ops
= let
rassoc, lassoc, nassoc :: [m (a -> a -> a)]
prefix, postfix :: [m (a -> a)]
....
in
termP <**> (rassocP <|> lassocP <|> nassocP <|> pure id)
<?> "operator"
In an equation for ‘buildExpressionParser’:
buildExpressionParser operators simpleExpr
= foldl makeParser simpleExpr operators
where
makeParser term ops
= let ...
in
termP <**> (rassocP <|> lassocP <|> nassocP <|> pure id)
<?> "operator"
splitOp (Infix op assoc) (rassoc, lassoc, nassoc, prefix, postfix)
= case assoc of
AssocNone -> ...
AssocLeft -> ...
AssocRight -> ...
splitOp (Prefix op) (rassoc, lassoc, nassoc, prefix, postfix)
= (rassoc, lassoc, nassoc, op : prefix, postfix)
splitOp (Postfix op) (rassoc, lassoc, nassoc, prefix, postfix)
= (rassoc, lassoc, nassoc, prefix, op : postfix)
|
115 | (rassoc,lassoc,nassoc,prefix,postfix) = foldr splitOp ([],[],[],[],[]) ops
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[4 of 8] Compiling Text.Parser.LookAhead ( src/Text/Parser/LookAhead.hs, dist/build/Text/Parser/LookAhead.o )
[5 of 8] Compiling Text.Parser.Permutation ( src/Text/Parser/Permutation.hs, dist/build/Text/Parser/Permutation.o )
[6 of 8] Compiling Text.Parser.Token.Highlight ( src/Text/Parser/Token/Highlight.hs, dist/build/Text/Parser/Token/Highlight.o )
[7 of 8] Compiling Text.Parser.Token ( src/Text/Parser/Token.hs, dist/build/Text/Parser/Token.o )
src/Text/Parser/Token.hs:642:38: warning: [-Wincomplete-uni-patterns]
Pattern match(es) are non-exhaustive
In a pattern binding: Patterns not matched: (_, [])
|
642 | (low, ex : high) = splitAt bd dps
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[8 of 8] Compiling Text.Parser.Token.Style ( src/Text/Parser/Token/Style.hs, dist/build/Text/Parser/Token/Style.o )
Preprocessing test suite 'doctests' for parsers-0.12.5..
Building test suite 'doctests' for parsers-0.12.5..
<no location info>: warning: [-Wmissing-home-modules]
These modules are needed for compilation but not listed in your .cabal file's other-modules: Build_doctests
[1 of 2] Compiling Build_doctests ( dist/build/doctests/autogen/Build_doctests.hs, dist/build/doctests/doctests-tmp/Build_doctests.o )
[2 of 2] Compiling Main ( tests/doctests.hs, dist/build/doctests/doctests-tmp/Main.o )
<no location info>: warning: [-Wmissing-home-modules]
These modules are needed for compilation but not listed in your .cabal file's other-modules: Build_doctests
Linking dist/build/doctests/doctests ...
Running 1 test suites...
Test suite doctests: RUNNING...
Test suite doctests: PASS
Test suite logged to: dist/test/parsers-0.12.5-doctests.log
1 of 1 test suites (1 of 1 test cases) passed.
And here's the contents of dist/test/parsers-0.12.5-doctests.log :
$ more dist/test/parsers-0.12.5-doctests.log
Test suite doctests: RUNNING...
Examples: 0 Tried: 0 Errors: 0 Failures: 0
-i
-i/home/rgscott/Documents/Hacking/Haskell/parsers-0.12.5/dist/build/autogen
-i/home/rgscott/Documents/Hacking/Haskell/parsers-0.12.5/dist/build
-i/home/rgscott/Documents/Hacking/Haskell/parsers-0.12.5/src
-hide-all-packages
-package-db=dist/package.conf.inplace
-optP-include
-optPdist/build/autogen/cabal_macros.h
-package-id=base-4.10.0.0
-package-id=base-orphans-0.6-9iPDLrUqQ5yHb9fb9OUZ9L
-package-id=charset-0.3.7.1-GkbAILH2Vj9HXMMGXPSeNg
-package-id=containers-0.5.10.2
-package-id=parsec-3.1.11-DPgnR92AWEaFOaixmwipet
-package-id=attoparsec-0.13.1.0-2xtipSJ3oFDBfLwedqccX
-package-id=text-1.2.2.2-EGUst8sqNAZCw1xLPcmcMH
-package-id=transformers-0.5.2.0
-package-id=mtl-2.2.1-19EL8AGBsN3DnnOhrC9xY3
-package-id=scientific-0.3.5.1-L0qYzdp0wpz8rO8gircNSR
-package-id=unordered-containers-0.2.8.0-HVcKYx0GQoVIIbKuoCcJbx
-package-id=bytestring-0.10.8.2
-package-id=directory-1.3.0.2
-package-id=doctest-0.12.0-JcuLvnckae744xvqjXdMWF
-package-id=filepath-1.4.1.2
-package=parsers-0.12.5
-package-id=QuickCheck-2.10.0.1-DTIBC3CyU6p3h4xSnOjkg1
-package-id=quickcheck-instances-0.3.16-A15GUudjPiB6fV2AyfxwM5
Text.Parser.Char
Text.Parser.Combinators
Text.Parser.LookAhead
Text.Parser.Permutation
Text.Parser.Expression
Text.Parser.Token
Text.Parser.Token.Style
Text.Parser.Token.Highlight
Test suite doctests: PASS
Test suite logged to: dist/test/parsers-0.12.5-doctests.log
And also
find dist?
$ find dist
dist
dist/package.conf.inplace
dist/package.conf.inplace/parsers-0.12.5-BbWYl7dZaXs10a39DfSMvp.conf
dist/package.conf.inplace/package.cache.lock
dist/package.conf.inplace/package.cache
dist/build
dist/build/libHSparsers-0.12.5-BbWYl7dZaXs10a39DfSMvp-ghc8.2.1.so
dist/build/Text
dist/build/Text/Parser
dist/build/Text/Parser/Token.dyn_hi
dist/build/Text/Parser/Combinators.o
dist/build/Text/Parser/LookAhead.o
dist/build/Text/Parser/Char.dyn_o
dist/build/Text/Parser/Permutation.dyn_o
dist/build/Text/Parser/Expression.dyn_o
dist/build/Text/Parser/Expression.dyn_hi
dist/build/Text/Parser/Permutation.o
dist/build/Text/Parser/Char.dyn_hi
dist/build/Text/Parser/Char.hi
dist/build/Text/Parser/LookAhead.dyn_o
dist/build/Text/Parser/Expression.o
dist/build/Text/Parser/Combinators.dyn_o
dist/build/Text/Parser/Token.hi
dist/build/Text/Parser/Char.o
dist/build/Text/Parser/Expression.hi
dist/build/Text/Parser/Permutation.hi
dist/build/Text/Parser/Token.o
dist/build/Text/Parser/Token
dist/build/Text/Parser/Token/Highlight.dyn_o
dist/build/Text/Parser/Token/Highlight.o
dist/build/Text/Parser/Token/Highlight.hi
dist/build/Text/Parser/Token/Style.dyn_hi
dist/build/Text/Parser/Token/Style.dyn_o
dist/build/Text/Parser/Token/Highlight.dyn_hi
dist/build/Text/Parser/Token/Style.hi
dist/build/Text/Parser/Token/Style.o
dist/build/Text/Parser/LookAhead.hi
dist/build/Text/Parser/Combinators.dyn_hi
dist/build/Text/Parser/Combinators.hi
dist/build/Text/Parser/LookAhead.dyn_hi
dist/build/Text/Parser/Permutation.dyn_hi
dist/build/Text/Parser/Token.dyn_o
dist/build/libHSparsers-0.12.5-BbWYl7dZaXs10a39DfSMvp.a
dist/build/autogen
dist/build/autogen/cabal_macros.h
dist/build/autogen/Paths_parsers.hs
dist/build/doctests
dist/build/doctests/autogen
dist/build/doctests/autogen/cabal_macros.h
dist/build/doctests/autogen/Build_doctests.hs
dist/build/doctests/autogen/Paths_parsers.hs
dist/build/doctests/doctests-tmp
dist/build/doctests/doctests-tmp/Main.o
dist/build/doctests/doctests-tmp/Build_doctests.hi
dist/build/doctests/doctests-tmp/Main.hi
dist/build/doctests/doctests-tmp/Build_doctests.o
dist/build/doctests/doctests
dist/setup-config
dist/cabal-config-flags
dist/test
dist/test/parsers-0.12.5.log
dist/test/parsers-0.12.5-doctests.log
dist/setup
dist/setup/Main.o
dist/setup/setup.version
dist/setup/Main.hi
dist/setup/setup
dist/setup/setup.hs
dist/setup/setup_macros.h
dist/hpc
dist/hpc/vanilla
dist/hpc/vanilla/tix
dist/hpc/vanilla/tix/doctests
rgscott@gearloose:~/.../Hacking/Haskell/parsers-0.12.5$ find dist
dist
dist/package.conf.inplace
dist/package.conf.inplace/parsers-0.12.5-BbWYl7dZaXs10a39DfSMvp.conf
dist/package.conf.inplace/package.cache.lock
dist/package.conf.inplace/package.cache
dist/build
dist/build/libHSparsers-0.12.5-BbWYl7dZaXs10a39DfSMvp-ghc8.2.1.so
dist/build/Text
dist/build/Text/Parser
dist/build/Text/Parser/Token.dyn_hi
dist/build/Text/Parser/Combinators.o
dist/build/Text/Parser/LookAhead.o
dist/build/Text/Parser/Char.dyn_o
dist/build/Text/Parser/Permutation.dyn_o
dist/build/Text/Parser/Expression.dyn_o
dist/build/Text/Parser/Expression.dyn_hi
dist/build/Text/Parser/Permutation.o
dist/build/Text/Parser/Char.dyn_hi
dist/build/Text/Parser/Char.hi
dist/build/Text/Parser/LookAhead.dyn_o
dist/build/Text/Parser/Expression.o
dist/build/Text/Parser/Combinators.dyn_o
dist/build/Text/Parser/Token.hi
dist/build/Text/Parser/Char.o
dist/build/Text/Parser/Expression.hi
dist/build/Text/Parser/Permutation.hi
dist/build/Text/Parser/Token.o
dist/build/Text/Parser/Token
dist/build/Text/Parser/Token/Highlight.dyn_o
dist/build/Text/Parser/Token/Highlight.o
dist/build/Text/Parser/Token/Highlight.hi
dist/build/Text/Parser/Token/Style.dyn_hi
dist/build/Text/Parser/Token/Style.dyn_o
dist/build/Text/Parser/Token/Highlight.dyn_hi
dist/build/Text/Parser/Token/Style.hi
dist/build/Text/Parser/Token/Style.o
dist/build/Text/Parser/LookAhead.hi
dist/build/Text/Parser/Combinators.dyn_hi
dist/build/Text/Parser/Combinators.hi
dist/build/Text/Parser/LookAhead.dyn_hi
dist/build/Text/Parser/Permutation.dyn_hi
dist/build/Text/Parser/Token.dyn_o
dist/build/libHSparsers-0.12.5-BbWYl7dZaXs10a39DfSMvp.a
dist/build/autogen
dist/build/autogen/cabal_macros.h
dist/build/autogen/Paths_parsers.hs
dist/build/doctests
dist/build/doctests/autogen
dist/build/doctests/autogen/cabal_macros.h
dist/build/doctests/autogen/Build_doctests.hs
dist/build/doctests/autogen/Paths_parsers.hs
dist/build/doctests/doctests-tmp
dist/build/doctests/doctests-tmp/Main.o
dist/build/doctests/doctests-tmp/Build_doctests.hi
dist/build/doctests/doctests-tmp/Main.hi
dist/build/doctests/doctests-tmp/Build_doctests.o
dist/build/doctests/doctests
dist/setup-config
dist/cabal-config-flags
dist/test
dist/test/parsers-0.12.5.log
dist/test/parsers-0.12.5-doctests.log
dist/setup
dist/setup/Main.o
dist/setup/setup.version
dist/setup/Main.hi
dist/setup/setup
dist/setup/setup.hs
dist/setup/setup_macros.h
dist/hpc
dist/hpc/vanilla
dist/hpc/vanilla/tix
dist/hpc/vanilla/tix/doctests
RyanGlScott commented
Note: there is no need to depend on the library, as we should pass everything to
docteststo load the package in GHCi. In that scenari we seem to forget something...
My hunch is that dist/build/autogen/cabal_macros.h only gets built when the library is built. If you only build the doctests test suite, then the only thing that gets built is dist/build/doctests/autogen/cabal_macros.h.
phadej commented
@RyanGlScott is it so, when there isn't dependency on parsers in test-suite, and you say cabal test doctest then the library isn't build?
phadej commented
@RyanGlScott exactly. So we don't really depend on the library, but we depend on the fact it should be configured (not built!)
I have to think about this.
phadej commented
I cannot reproduce this. Which cabal-install do you use?
RyanGlScott commented
I've reproduced this with both 2.0 and 1.24.
RyanGlScott commented
Make sure you're doing this from a completely clean build, since the presence of dist/build/autogen/cabal_macros.h will affect whether you can trigger the error or not.
phadej commented
I see. I cannot reproduce it with GHC-8.0.2, but I can with GHC-8.2.1
phadej commented
ezyang commented
CC @angerman who contributed the doctest feature. It's probably just a simple bug. Oops, this is a bug on cabal-doctest, not the new native doctest support in Setup.
ezyang commented
The most likely reason why this only repros on GHC 8.2 is because GHC 8.2 is bundled with Cabal 2.0, which correctly generates separate cabal_macros.h for each component (and not just a single cabal_macros.h for the entire package.) So it is probably a bug in cabal-doctest where it is looking for the library macro, when it really should not: it should look at the component specific macro. See haskell/cabal#1893
RyanGlScott commented
FWIW, I also recall experiencing this issue (having the test suites build before the library after invoking cabal test, where the test suites don't list the library as a dependency) when upgrading singletons to use Cabal-2.0, and singletons doesn't use a custom setup script. (See goldfirere/singletons#182 (comment).) At the time, I gave up and just made the test suite list the singletons library as an explicit dependency.