Creating a Full Haskell Stack with Tests - haskell

I'm new to Haskell and I'm trying to structure a program under test. I have decided to use HUnit and Cabal.
From what I have seen a well strucutred project looks the following:
src/
AppName/
Appname.hs
testsuite/
tests/
AppName/
TestRunner.hs
AppName.cabal
Setup.hs
The parts that are a mystery to me are the TestRunner.hs and the AppName.cabal.
What would a testrunner look like that runs all of the test under the testsuite/tests directory and sub directories? And how can it be integrated with Cabal?
Also, how do you put the hackage dependencies in the AppName.cabal and build them from the command line?
I am having a hard time finding a full example building an application from scratch with tests and dependencies.
Thanks

Here's a fragment of the .cabal file I used for one of my recent libraries.
...
Library
Build-depends: base >= 4 && < 5, bytestring, directory, filepath, hslogger,
SHA, zlib
Ghc-options: -Wall
Exposed-modules: Ltc.Store
Test-suite reference
Hs-Source-Dirs: Test, .
Main-Is: ReferenceProps.hs
Type: exitcode-stdio-1.0
Build-Depends: base >= 4 && < 5, bytestring, directory, filepath, hslogger,
SHA, zlib
Ghc-Options: -Wall
Build-Depends: test-framework, test-framework-hunit, test-framework-quickcheck2,
HUnit, QuickCheck
As we can see the cabal file defines a library and a testsuite. The library defines the modules it exports, the packages it depends on, and sets some custom GHC options.
We can easily build and package the library for distribution with:
% cabal configure
% cabal build
% cabal sdist
The testsuite looks a lot like the the library: first off, it has the same dependencies as the library (see the first Build-Depends line), and it then adds some extra test dependencies (see the second Build-Depends line). The testsuite here is a combination of HUnit and QuickCheck tests, and it uses Test-Framework as the runner. The test proper is Test/ReferenceProps.hs. It's a exitcode-stdio type test. This means that cabal will say that the tests pass if ReferenceProps exits with code 0. Otherwise, it will say the tests failed.
The testsuite looks like this (but, here, we're going to use some simple tests for list reversals):
import Data.Monoid
import Test.Framework
import Test.Framework.Providers.HUnit
import Test.Framework.Providers.QuickCheck2
import Test.HUnit
import Test.QuickCheck
main :: IO ()
main = defaultMainWithOpts
[ testCase "rev" testRev
, testProperty "listRevRevId" propListRevRevId
] mempty
testRev :: Assertion
testRev = reverse [1, 2, 3] #?= [3, 2, 1]
propListRevRevId :: [Int] -> Property
propListRevRevId xs = not (null xs) ==> reverse (reverse xs) == xs
The main is just a harness. You can also set various options for test-framework by replacing the mempty. The function testRev is a HUnit test, and propListRevRevId is a QuickCheck test; see the relevant docs on how to write these.
Finally, we can run the tests:
% cabal configure --enable-tests
% cabal test

Related

Automatically running unit tests in Haskell

Is there a way to run my test suites with my build command? All I can find is that I can run e.g. cabal test after cabal build. What I want is a way to automatically run my tests when running e.g. cabal build.
I want a solution that will automate this for anyone building the package without requiring them to run a different script. An analogy would be a java build script where the default build target runs unit tests after compiling.
You can use a cabal.project.local to do this when using cabal v2 projects (as in, cabal v2-build which is the default and thus the same as cabal build for cabal version 3.0 and up).
The strategy is just to place a cabal.project.local file in the build directory with contents of:
tests: true
I find this a really bad idea since it's unexpected, non-canonical behavior that breaks cross-compilation, but you can probably hack it together with a Custom build type.
Here's an example mypackage.cabal:
cabal-version: >=2.0
name: mypackage
version: 0.1.0.0
author: None
maintainer: none#example.com
build-type: Custom
custom-setup
setup-depends:
base >= 4.5,
Cabal >= 2.0
executable mypackage
main-is: Main.hs
build-depends: base >= 4
default-language: Haskell2010
test-suite test
type: exitcode-stdio-1.0
build-depends: base >= 4
main-is: Test.hs
and a corresponding Setup.hs:
import Control.Monad
import Distribution.Simple
import Distribution.Simple.Setup
import Distribution.Simple.Test
import Distribution.Simple.Build
main = defaultMainWithHooks simpleUserHooks {
postBuild = myPostBuild
}
myPostBuild args flags desc info =
when ("exe:mypackage" `elem` buildArgs flags) $ do
build desc info defaultBuildFlags { buildDistPref = buildDistPref flags } []
test ["test"] desc info defaultTestFlags {
testDistPref = buildDistPref flags
}
This does result in a project where cabal build also builds and runs the tests on my Cabal 3.0.0.0 system, but I don't know which additional caveats this has.

Set up Haskell Project and run tests

I'm trying to set up a Haskell project (library) with tests that I can use to work through The Haskell Road to Logic, Maths, and Programming. There are three parts that I'd like to have:
The code that comes with the book, in a subdirectory
The code I write for exercises in the book; one file per chapter
The code I write for tests; one file per chapter
I have attempted a project setup here, but am getting the following cabal error:
Resolving dependencies...
Configuring haskell-road-0.1.0.0...
Building haskell-road-0.1.0.0...
Failed to install haskell-road-0.1.0.0
Build log ( /Users/stuart/.cabal/logs/haskell-road-0.1.0.0.log ):
cabal: Entering directory '.'
Configuring haskell-road-0.1.0.0...
Building haskell-road-0.1.0.0...
Preprocessing library haskell-road-0.1.0.0...
src/Chapter1.hs:1:1:
File name does not match module name:
Saw: ‘Main’
Expected: ‘Chapter1’
cabal: Leaving directory '.'
cabal: Error: some packages failed to install:
haskell-road-0.1.0.0 failed during the building phase. The exception was:
ExitFailure 1
I'd like to be able to run $ cabal test and have all of the tests run, and have the import paths work. Any help is appreciated. I think there are probably issues with the test structure, but I've had trouble finding definitive guides on the actual setup.
EDIT: More details
src/
Chapter1.hs
Book/
GS.hs
etc....
test/
Chapter1Test.hs
MainTestSuite.hs
TestHelper.hs
haskell-book.hs:
-- Initial haskell-road.cabal generated by cabal init. For further
-- documentation, see http://haskell.org/cabal/users-guide/
-- The name of the package.
name: haskell-road
-- The package version. See the Haskell package versioning policy (PVP)
-- for standards guiding when and how versions should be incremented.
-- https://wiki.haskell.org/Package_versioning_policy
-- PVP summary: +-+------- breaking API changes
-- | | +----- non-breaking API additions
-- | | | +--- code changes with no API change
version: 0.1.0.0
-- A short (one-line) description of the package.
-- synopsis:
-- A longer description of the package.
-- description:
-- The license under which the package is released.
license: MIT
-- The file containing the license text.
license-file: LICENSE
-- The package author(s).
author: Stuart Terrett
-- An email address to which users can send suggestions, bug reports, and
-- patches.
maintainer: shterrett#gmail.com
-- A copyright notice.
-- copyright:
-- category:
build-type: Simple
-- Extra files to be distributed with the package, such as examples or a
-- README.
extra-source-files: ChangeLog.md
-- Constraint on the version of Cabal needed to build this package.
cabal-version: >=1.10
library
-- Modules exported by the library.
exposed-modules: Chapter1, Book.COR, Book.DB, Book.FAIS, Book.FCT, Book.GS, Book.Hierarchy, Book.IAR, Book.Nats, Book.POL, Book.Polynomials, Book.PowerSeries, Book.Query, Book.REL, Book.SetEq, Book.SetOrd, Book.STAL, Book.TAMO, Book.TUOLP, Book.WWN
-- Modules included in this library but not exported.
-- other-modules:
-- LANGUAGE extensions used by modules in this package.
other-extensions: FlexibleInstances
-- Other library packages from which modules are imported.
build-depends: base, random >=1.1 && <1.2, HUnit >=1.3 && <1.4
-- Directories containing source files.
hs-source-dirs: src
-- Base language which the package is written in.
default-language: Haskell2010
test-suite haskell-road-tests
type: exitcode-stdio-1.0
hs-source-dirs: tests, src
main-is: MainTestSuite.hs
build-depends: base,
HUnit,
QuickCheck,
test-framework,
test-framework-hunit,
test-framework-quickcheck2
MainTestSuite.hs
import Chapter1Test
exitProperly :: IO Counts -> IO ()
exitProperly m = do
counts <- m
exitWith $ if failures counts /= 0 || errors counts /= 0 then ExitFailure 1 else ExitSuccess
allTests::[Test]
allTests = [Chapter1Test.itRuns]
main :: IO ()
main = exitProperly (runTestTT (TestList allTests))
Diff of all changes:
http://lpaste.net/5997592404872396800
Specific changes you need to make:
In Chapter1.hs make sure module Chapter1 appears before the import statement:
module Chapter1 where
import ...
In each of the Book modules you need to add the prefix Book. to
each of the module statements, e.g. in Book/COR.hs:
change: module COR
to: module Book.COR
Also, any import statement will also need the Book. prefix, i.e. in Book/STAL.hs:
change: import DB
to: import Book.DB
(It might easier just to leave the book's modules at the top-level of the module name space.)
To fix this compilation error:
src/Book/IAR.hs:131:7:
No instance for (Foldable t3) arising from a use of ‘foldr’
just add {-# LANGUAGE NoMonomorphismRestriction #-} to the top of Book/IAR.hs (it should be the very first line.)
To fix this compilation error:
src/Book/FAIS.hs:14:4: Parse error in pattern: n + 1
change: f (n+1) = True : f n to f n = True : f (n-1).
This is called an n+k pattern and more info about it (and why it has been deprecated) is available here: What are "n+k patterns" and why are they banned from Haskell 2010?
In the test-suite section you have:
hs-source-dirs: tests, src
To use the code in the src directory you tests should depend on the haskell-road library instead of compiling the source code. That is, use these lines in the test-suite section:
hs-source-dirs: tests
build-depends: base, haskell-road, HUnit, ...
File test/Chapter1Test.hs needs a module statement:
module Chapter1Test where
and also fix this import statement:
-import TestHelper.testCase
+import TestHelper (testCase)
File test/MainTestSuite.hs needs these import statements:
import System.Exit
import Test.HUnit
File test/testHelper.hs needs to be renamed to test/TestHelper.hs
and also needs this import statement:
import Test.HUnit
Cabal has
developing packages link and cabal file content structure is described there.
By looking the error message, it seems that your haskell library source file Chapter1 starts with module Main where. It should contain module Chapter1 where, as the error message says.
Libraries should not contain main while the test-executables should, which is why you state in the cabal file the test executables with main-is.
Hope this helps! (I didn't look at the github sources, just the error message.)

hspec failing to import (private) code dependency despite CPP override

Let's say I have a src file like so:
{-# LANGUAGE CPP #-}
module Alphabet (
#ifdef TEST
alphabet
#endif
) where
alphabet :: [Char]
alphabet = "abcdefghijklmnopqrstuvwxyz"
a .cabal file like so:
name: Alphabet
version: 0.1.0.0
library
build-depends: base >=4.8 && <4.9, containers >=0.5 && <0.6, split >=0.2 && <0.3
hs-source-dirs: src
Exposed-modules: Alphabet
default-language: Haskell2010
test-suite alphabet-test
ghc-options: -Wall -Werror
cpp-options: -DTEST
default-extensions: OverloadedStrings
type: exitcode-stdio-1.0
main-is: Spec.hs
hs-source-dirs: tests
build-depends: Alphabet, base >= 4.8 && < 4.9, containers >= 0.5 && <0.6, split >= 0.2 && < 0.3, hspec, QuickCheck
default-language: Haskell2010
A master test file like so:
{-# OPTIONS_GHC -F -pgmF hspec-discover #-}
and a test file like:
module AphabetSpec (spec) where
import Test.Hspec
import Alphabet (alphabet)
spec :: Spec
spec = do
describe "Alphabet.alphabet" $ do
it "returns the alphabet" $ do
alphabet `shouldBe` "abcdefghijklmnopqrstuvwxyz"
now running `cabal test`:
cabal test
Preprocessing library Alphabet-0.1.0.0...
In-place registering Alphabet-0.1.0.0...
Preprocessing test suite 'alphabet-test' for Alphabet-0.1.0.0...
[1 of 1] Compiling Main ( tests/AlphabetSpec.hs, dist/build/alphabet-test/alphabet-test-tmp/AlphabetSpec.o )
tests/AlphabetSpec.hs:4:27:
Module ‘Alphabet’ does not export ‘alphabet’
Why is my CPP not working as expected? How can I fix it?
Why is my CPP not working as expected?
Two-step building. Since your tests depend on the library, it gets build first. The library doesn't have any CPP options set, therefore alphabet doesn't get exported.
When the tests get built, your library is already compiled, and alphabet doesn't get exported. It's a separation of concerns.
How can I fix it?
There are several tricks to work with "hidden" (e.g. non-exported) functions. For one, you can put them all into a .Internal module. That way, users that want to use the hidden bits can do so rather easily, but the casual user doesn't have too many tools at hand.
Another way to handle this is to drop the dependency in the test and instead add the src directory to the tests:
hs-source-dirs: tests, src
However, this also means that you have to rebuild the whole library for tests, but it will enable using CPP.
A third option is to not test alphabet, but instead the observable behaviour of exported functions that depend on it. So instead of testing alphabet, you would test filterAlpha:
filterAlpha :: String -> String
filterAlpha = filter (`elem` alphabet)
You have to test filterAlpha anyway. If there are many functions that use alphabet, it is likely that you will have some test that will notice a regression if you accidentally change it.
The problem was a simply syntax error. #ifdef is wrong, #ifndef is right

Testing Haskell code defined in the Main module

I'm looking for help regarding how to write tests for Haskell code that is defined in the Main module.
The project I want to test is a cabal package in which I've defined multiple executables.
Each executable code is declared only made of a single file (one for each project euler problem), and I usually run them individually with the cabal run command.
I have tried to write a test, also in the Main module, but when compiling, the function I'm trying to test is not found ("Not in scope" error).
What is the right way to write tests in this case?
For information, below is the directory layout of my project:
pe/ # root
pe.cabal
src/
Util.hs
Problem001.hs # "module Main where" and declares a main function
Problem002.hs # "module Main where" and declares a main function
(...)
test/
TestProblem001.hs # "module Main where" and declares a main function
Below is an extract from pe.cabal:
test-suite test-all
hs-source-dirs: test
type: exitcode-stdio-1.0
main-is: TestProblem001.hs
build-depends: base, HUnit, Cabal >= 1.9.2
executable problem-001
hs-source-dirs: src
main-is: Problem001.hs
build-depends: base
ghc-options: -Wall -Werror -O2
[edit]
As I couldn't find any resource for this exact requirement, I opted for a project architecture which is easier to test: problems are defined as library and not individual executables anymore.
I do so
test1 = ...
test2 = ...
main = do
args <- getArgs
case args of
... -> check test1
... -> check test2

plugins package unknown symbol when using cabal

I'm messing around with the plugins package however I bumped into a problem.
Here's the code:
Util/Header.hs
module Util.Header(PT(..)) where
data PT a = PT a deriving Show
Plug.hs
module Plug(helloPlugin) where
import Util.Header
helloPlugin :: PT Int
helloPlugin = PT 1
Main.hs
module Main where
import Util.Header
import System.Plugins
main :: IO ()
main = do
mv <- load "Plug.o" ["."] [] "helloPlugin"
case mv of
LoadFailure msg -> print msg
LoadSuccess _ v -> print $ show (v :: PT Int)
This all works fine then compiling with ghc. Building with Cabal works fine as well, but when I run the executable I get this error:
plugintest: /home/kevin/.cabal/lib/plugins-1.5.4.0/ghc-7.6.3/HSplugins-1.5.4.0.o: unknown symbol `ghczm7zi6zi3_ErrUtils_zdsinsertzuzdsgo5_info'
plugintest: user error (resolvedObjs failed.)
My very minimalistic cabal file:
name: plugintest
version: 0.1.0.0
license-file: LICENSE
build-type: Simple
cabal-version: >=1.8
library
hs-source-dirs: src
exposed-modules: Util.Header
build-depends: base ==4.6.*, plugins ==1.5.*
executable plugintest
main-is: Main.hs
build-depends: base ==4.6.*, plugins ==1.5.*, plugintest == 0.1.0.0
hs-source-dirs: src
Now I assume the problem is that it can't find the "ErrUtils" module which is part of the ghc package installed in /usr/lib/ghc-7.x.x.
Since it's using cabal it'll use the $HOME/.cabal/lib/ instead.
Now I obviously wouldn't want to use /usr/lib if I wanted to make it distributable. Sadly I'm not very familiar with how packages are managed nor am I familiar with the plugins package.
I have a feeling this is extremly nooby but I wasn't able to find a solution myself.
So a few questions:
How can I get my dependencies to work in a way to make this distributable?
It seems I'll need to know beforehand what my Plugin.o files will depend on before actually being able to use them (If I understand correctly).
Is there a way to package a .o files that I wouldn't have to worry about this problem? (Sorry if this question is too vague, feel free to ignore)
Thanks in advance!
Ok, so I had the exact same problem.
Here is a workaround I found
Change the load call to
load "Plug.o" [".","dist/build/plugintest/plugintest-tmp"] [] "testplugin"
Make sure you compile the thing with -c or by using the "make" library from plugins.
Quite annoyed by this... The error suggests it is having issues linking against the standard libs, so why does showing it these .o files fix it?
Anyways, this worked for me, and didn't require a ton of mucking around with .cabal files.
You must declare your exported- and other- modules in order for Cabal to package them all together. For instance (from https://github.com/tel/happstack-heroku-test)
name: hktest -- note the name here names
-- the *library* which is a package name
-- in scope when building the executable
...
library
exposed-modules:
HKTest
other-modules:
-- there aren't any, but there could be some
build-depends: base >= 4.6 && <4.7
...
, mtl >= 2.1.2
hs-source-dirs: src
executable server
main-is: Server.hs
other-modules:
-- there might be some use to having these here,
-- but they'll be harder to get into GHCi, so I wouldn't
-- recommend it---just put them in the library part
build-depends: base >=4.6 && <4.7
, hktest -- note that I grab all the hktest
-- modules here
hs-source-dirs: exe
If I leave out one of those modules I'll likely get a build error as Cabal compiles files which expect to be able to find symbols that haven't been packaged.
In your case, since you're building an executable, the common pattern exemplified above is to put all of your code into a library and then have the executable side depend upon that library. For instance, in this example the complete text of exe/Server.hs is
module Main where
import qualified HKTest as HK
main :: IO ()
main = HK.main

Resources