I want to generate code coverage report using Stack. I run command that
amounts to (omitting options passed to test suite via --test-arguments):
$ stack test --coverage
This performs the testing and then outputs the following:
Error: The coverage report for myproject's test-suite "tests" did not
consider any code. One possible cause of this is if your test-suite builds
the library code (see stack issue #1008). It may also indicate a bug in
stack or the hpc program. Please report this issue if you think your
coverage report should have meaningful results.
I think it should (this creates empty report). GHC options are identical for
all components of my package. There is no need for test suite to rebuild the
library. After all, if Cabal can generate the report, Stack should be able
to do it given the same Cabal config or am I mistaken?
I've opened
an issue on Stack
GitHub repo as suggested.
After a while I decided to create good old sandbox and generate the report
using Cabal instead (I really need to see the report, you know). It worked
previously, but now I get:
$ cabal sandbox init
… <everything OK>
$ cabal update
… <everything OK>
$ cabal install --only-dependencies --enable-tests
… <everything OK>
$ cabal configure --enable-tests --enable-coverage
… <everything OK>
$ cabal build
… <everything OK>
$ cabal test
Running 2 test suites...
Test suite tests: RUNNING...
Test suite tests: PASS
Test suite logged to: dist/test/myproject-0.1.0-tests.log
hpc: can not find HUnit_DDLSMCRs3jyLBDbJPCH01j/Test.HUnit.Lang in ["./.hpc","./dist/hpc/vanilla/mix/myproject-0.1.0","./dist/hpc/vanilla/mix/tests"]
What? I've never seen this, although I generated many reports
before. Someone up there just decided that I won't get that report today,
it seems.
Do you know how to generate coverage report using Stack? Has anyone
succeeded at this?
In my case I was still getting this error. Running:
stack clean
stack test --coverage
solved the problem, as reported here.
Recent changes upstream fixed it. Should be resolved for users of 0.1.7.0 and later.
Related
I wanted to get going with Haskell a little bit and therefore took a look at the Spock framework. To start clean, I uninstalled everything Haskell related from my Arch Linux machine and installed ghcup, Cabal and Stack using the install scripts from their respective websites.
Now I want to follow Spock's Tutorial. Trying to install Spock globally with cabal install Spock as suggested gives me an error (abbreviated):
src/Web/Spock/Internal/Wire.hs:43:1: error:
Could not find module ‘Web.Routing.AbstractRouter’
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
|
43 | import Web.Routing.AbstractRouter
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cabal: Failed to build Spock-0.9.0.1. See the build log above for details.
I already found a question on reddit on the topic, but the solution does not apply because I'm not trying to use a specific version of the libraries as implied.
So I try to follow along and build only locally.
But when I reach the point where it says stack build --fast --pedantic, the build plan can not be constructed and Stack suggests to add another dependency, stm-containers. Doing so, I am presented with two additional suggestions for focus and primitive. When I add these, the plan fails again, this time without a simple solution:
In the dependencies for primitive-0.6.4.0:
base-4.13.0.0 from stack configuration does not match >=4.5 && <4.13 (latest matching version is 4.12.0.0)
needed due to Spock-example-0.1.0.0 -> primitive-0.6.4.0
I can do a little thing with Haskell, but with the build system(s), I am way out of my comfort zone. Help and hints appreciated. Oh, and all versions of course are the latest by the time of this post.
Due to incompatible versions of dependencies, Spock won't build with GHC 8.8 and above. A similar problem is described in Spock issue #149, though I'm not fully sure it is exactly the same incompatibility. The error you got from Stack hints at that, as base-4.13.0.0 is the version of base that is bundled with GHC 8.8. cabal-install failed in a more obscure way because, upon noting the incompatibility, it tries to solve the dependencies using older versions of Spock, eventually picking 0.9.0.1, attempting and, thanks to a missing version upper bound for the reroute dependency, failing to build it.
(Shortly after this answer was posted, the missing upper bound was retrofitted to the old Spock version, so attempting to reproduce the problem now will lead to an easier to understand failure.)
Casting the tutorial aside, the most straightforward way to use Spock given those complications is probably through cabal-install 3+. Begin by using ghcup to switch to GHC 8.6.5:
$ ghcup install 8.6.5
$ ghcup set 8.6.5
Then, create a blank project with cabal-install:
$ mkdir myproject
$ cd myproject
$ cabal init
Add Spock to the build-depends section of myproject.cabal:
build-depends: base >=4.12 && <4.13
, Spock == 0.13.*
Finally, you can run:
$ cabal build
Which will install Spock and its dependencies before building the project. (Note that you generally don't need to use cabal install to install libraries with cabal-install 3.)
It is presumably possible to make it work with Stack as well, by changing to the lts-14.27 resolver (the latest one that uses GHC 8.6.5), tracking down all dependency versions that need to be overriden (as you had began to do) and manually adding them to the extra-deps of stack.yaml.
I recently upgraded to Cabal 3.2 (and GHC 8.10) and I am running into some major issues that make some of my project non-buildable anymore...
Thorough description of the problem
Here is a minimal (not) working configuration that fails every time:
I start off with a clean Cabal configuration (by deleting ~/.cabal); the reason for that will appear later in the post. I run cabal update to recreate the .cabal directory and to ensure Cabal is working.
I create a project (let's call it test1) using cabal init. This is a library project with one exposed module (conveniently named Test1) that exports some dummy function foo. I run cabal build, then cabal install --lib; everything is running smooth, so far so good.
Just to be sure, I leave the project directory and fire up GHCi. I type in :m Test1 to load the module I created earlier, and it works! I can type in foo ... and see my function executed. Also, I list the content of ~/.cabal/store/ghc-8.10.xxx and see that the test1-xxx directory is there.
I then create a new project, test2, still using cabal init. This time, I configure it to be an executable, and I add test1 as a dependency (using the build-depends field). But this time when I run cabal build, I run into some issue:
~/projects/haskell/test2> cabal build
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: test2-0.1.0.0 (user goal)
[__1] unknown package: test1 (dependency of test2)
[__1] fail (backjumping, conflict set: test1, test2)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: test2, test1
It seems to me like package test1 cannot be found, however I can access it from GHCi (and GHC for that matters) and it is present in ~/.cabal/store...
But unfortunately there is more.
I create a third project, test3. This is a library, and it depends on nothing else than base (so in particular it does not depend on test1). The lib exposes one module, Test3, with one function exported, bar. I run cabal build, no problem here. But when I want to install test3 with cabal install --lib I run into some errors:
~/projects/haskell/test3> cabal install --lib
Wrote tarball sdist to
/home/<user>/projects/haskell/test3/dist-newstyle/sdist/test3-0.1.0.0.tar.gz
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] unknown package: test1 (user goal)
[__0] fail (backjumping, conflict set: test1)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: test1
It seems that it cannot find test1, although it has been installed correctly; may be this is a remnant of the failed build of test2 though...
Just to be sure, I fire up GHCi and type in :m Test3, but GHCi tells me that it cannot find module Test3 (and even suggests this is a typo and I was meaning Test1), showing that test3 indeed did not get installed, although it got successfully built...
Okay there is one more quirk to this whole situation: I create once again a new project with cabal init, called test4, which is an executable that (again) depends on nothing else than base. I keep the default Main.hs (that just prints "Hello, Haskell!"). I run cabal build: no problem. Then I run cabal install and... No problem either? I run test4 in a random location and it fires up the executable, printing "Hello, Haskell!" in the terminal...
And there is one last thing: I go to some random location and I run cabal install xxx --lib where xxx is a library package available on Hackage (for example xml) and:
~> cabal install xml --lib
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] unknown package: test1 (user goal)
[__0] fail (backjumping, conflict set: test1)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: test1
This is the reason why I need to nuke .cabal regularly... Right now I seem to be in some kind of stale state where I cannot install any library anymore.
Technical configuration and notes
I am running Cabal 3.2.0.0 and GHC 8.10.0.20200123. I installed them from the hvr/ghc PPA, and I made sure there are no other versions of those tools anywhere on my computer.
Just as a note, I am running Ubuntu 18.04.4 LTS (with XFCE so XUbuntu to be exact). Everything else (seem to be like it) is up to date.
Last thing, regarding the *.cabal files I use for building, they are pretty much the ones generated by cabal init, except I switch executable xxx for library in the case of libraries, and I simply add a exposed-modules field for exposing modules for the libraries (so Test1 for test1 and Test3 for test3 respectively). I also use build-depends in test2 to make the project depend on test1. Apart from that, they are pretty much left untouched.
Notes and thoughts
I must confess that I am new to Cabal 3; until last week I was using Cabal 1 (because I never bothered to update it; yes I know this is bad). With Cabal 1 I did not have any problem whatsoever, and I was perfectly able to install a package from local sources and depend on it in other projects...
I feel like I am doing something wrong; maybe am I not using the correct Cabal commands? I saw somewhere something about cabal new-build and cabal new-install but it does not seem to do anything more than cabal build and cabal install, at least in my case. I also wanted to investigate sandboxes but it seems that has disappeared since version 2 of Cabal.
There is also a slight possibility this is a Cabal bug, but I don't find any relevant issue on the bug tracker that may be related to my problem...
What do you think about this? What am I doing wrong? Do you see any alternative or possible fix?
Thanks a lot!
GHC environment files
A GHC installation comes with a certain number of packages out-of-the box. base is one of them but there are others, for example text. If you install GHC alone (no cabal or stack) and open ghci, it should let you import Data.Text without problems.
What if you want GHC or ghci to be aware of other compiled packages present in your filesystem? You can point GHC to additional package databases using command-line flags, but there's also the concept of package environment files.
Environments are plain text files that contain a list of package-related GHC flags. There might be a global environment at ~/.ghc/$ARCH-$OS-$GHCVER/environments/default, and there might also exist local environments which only affect GHC and ghci commands invoked inside the same folder. The exact rules for search are described in the GHC User Guide.
What does cabal install --lib actually do?
By default, it modifies the global environment file, so that GHC and ghci can now find that library. That's why point 3) worked. The actual compiled binaries of the library still reside in the cabal store though.
We can also create local environment files. For example cabal install sop-core --lib --package-env . will create the environment file .ghc.environment.xxx in the current folder, and the library will be available to ghc and ghci when they are invoked there.
Why isn't test1 available for test2?
Modern cabal makes a distinction between local packages and external packages.
local packages is the set of packages you are developing together in a project, being edited, recompiled and changed repeatedly. They are built "inplace" and not seen outside the project. They can depend on each other.
external packages are dependencies from build-depends: whose source code is downloaded from a package repository and which, when compiled, are put in the cabal store so that other Cabal projects might make use of them without re-compiling.
The list of local packages and other project-level configuration details are specified in a cabal.project file. But you don't need one if you work on a single isolated package; the default list of packages is simply ./*.cabal.
cabal wants to completely control the build environment of local packages, and will ignore the global environment file. In your case, you'll have to make test1 and test2 local packages in the same project (likely the best option) or publish test1 and treat it as an external package.
Note that "cabal project" is a concept relevant only during development. Packages are published independently, there are no "projects" in Hackage or other repositories, just packages.
What if I want to treat test1 as external without publishing it to Hackage?
You will have to set up a local package repository, basically a non-public Hackage.
You can tell Cabal about additional package repositories in the Cabal configuration file, that is, the file that configures cabal itself. Its location is given in the last line of cabal --help.
But how to set up the repository? The hackage-repo-tool can help with that.
Why did test3 fail? Why did further library installs fail?
That's weird, I have no idea why that happens. Did you by perchance delete the ~/.cabal folder between steps 3) and 5) ? What happens if you delete the global GHC environment file and try again?
question:
What do I have to do in the .cabal file in order to cause the libraries to link rather than build?
background:
I am trying to get coverage details from the command stack test --coverage
when I run this build I get the error message
Error: The coverage report for xmonad's test-suite "properties" did not consider any code. One possible cause of this is if your test-suite builds the library code (see stack issue #1008). It may also indicate a bug in stack or the hpc program. Please report this issue if you think your coverage report should have meaningful results.
Only one tix file found in /home/paul/temp/xmonad_coverage/.stack-work/install/x86_64-linux/09c83ca90bc1875ad3d1b5ea4a2a0c369c6367f3ad989533e627c073ee9962e0/8.0.1/hpc/, so not generating a unified coverage report.
on the stack documentation site, https://docs.haskellstack.org/en/stable/coverage/, it says that in order to run coverage I must have :
These test-suites link against your library, rather than building the
library directly. Coverage information is only given for libraries, ignoring
the modules which get compiled directly into your executable. A common case
where this doesn't happen is when your test-suite and library both have
something like hs-source-dirs: src/. In this case, when building your test-
suite you may also be compiling your library, instead of just linking against
it.
when I look in my .cabal file for the library there is
hs-source-dirs: src
and for test there is.
hs-source-dirs: tests
I don't understand the purpose of these and whether these are causing the library to build rather than be linked to.
could this be the reason that stack test --coverage is failing? or am I looking in the wrong place?
Turns out that I could fix this by doing the following:
stack clean; stack build; stack test --coverage --ghc-options "-fforce-recomp"
as described by Michael Sloan at https://github.com/commercialhaskell/stack/issues/1305
his explanation for a similar problem was:
Looks like what's happening is that the library isn't getting rebuilt, despite
reconfiguring with --ghc-options -fhpc and building the package. As a result, the .tix
file generated by the test only includes coverage info for the test itself
In the old cabal, you could pass test options like --color and --match=name by doing cabal test --test-option=--color --test-option=--match=name. Can this be done with cabal new-test? I don't see a --test-option in the help output of cabal new-test --help.
No, it is not possible, presumably because new-test runs all test suites, so it is unclear which test to pass it to.
But new versions of cabal tell you in the help for new-test:
To pass command-line arguments to a test suite, see the new-run command.
so that is now the way to run a single test suite with options.
I've made a very simple project with a failing test suite https://github.com/k-bx/noruntests-play
Now when I run stack --test --no-run-tests build I would expect it to build the project, but not run the tests. Instead, it runs the tests:
➜ noruntests-play git:(master) stack --test --no-run-tests build
noruntests-play-0.1.0.0: test (suite: test)
test: error
CallStack (from HasCallStack):
error, called at tests/Tests.hs:4:8 in main:Main
Test suite failure for package noruntests-play-0.1.0.0
test: exited with: ExitFailure 1
Logs printed to console
What am I doing wrong here? Thank you!
You should put build option before --test like this:
$ stack build --test --no-run-tests
I'm not sure whether it's bug or feature. You can open issue here if you're interested in feedback from developers. Personally for me it seems strange to pass --test before build. In some reasonable sense --test is subpart of build and subparts or options are usually specified to the right of main option.
Also there's shorter version of what you want (because build --test is just test):
$ stack test --no-run-tests