Running all combinations of tests and test cases in Haskell - haskell

Question

I aim to be able to define a collection of test methods and a collection of test cases (input/output data) and then execute all of their combinations. The goal is to avoid re-writing the same code over and over again when having say, 3 different implementations of the same function and 4 test cases that the function should satisfy. A naive approach would require me to write 12 lines of code:
testMethod1 testCase1
testMethod1 testCase2
...
testMethod3 testCase4
I've a gut feeling that Haskell should provide a way to abstract this pattern somehow. The best thing I've currently came up with is this piece of code:
import Control.Applicative
data TestMethod a = TM a
data TestData inp res = TD inp res
runMetod (TM m) (TD x res) = m x == res
runAllMethods ((m, inp):xs) = show (runMetod m inp) ++ "\n" ++ runAllMethods xs
runAllMethods _ = ""
head1 = head
head2 (x:xs) = x
testMethods = [TM head1, TM head2]
testData = [TD [1,2,3] 1, TD [4,5,6] 4]
combos = (,) <$> testMethods <*> testData
main = putStrLn $ runAllMethods combos
This works, computes 2 tests against two 2 functions and prints out 4 successes:
True
True
True
True
However, this works only for lists of the same type, even though the head function is list type agnostic. I would like to have a test data collection of any lists, like so:
import Control.Applicative
data TestMethod a = TM a
data TestData inp res = TD inp res
runMetod (TM m) (TD x res) = m x == res
runAllMethods ((m, inp):xs) = show (runMetod m inp) ++ "\n" ++ runAllMethods xs
runAllMethods _ = ""
head1 = head
head2 (x:xs) = x
testMethods = [TM head1, TM head2]
testData = [TD [1,2,3] 1, TD ['a','b','c'] 'a']
combos = (,) <$> testMethods <*> testData
main = putStrLn $ runAllMethods combos
but this fails with an error:
main.hs:12:21: error:
No instance for (Num Char) arising from the literal ‘1’
In the expression: 1
In the first argument of ‘TD’, namely ‘[1, 2, 3]’
In the expression: TD [1, 2, 3] 1
Is it possible to achieve this test-function X test-case cross testing somehow?

Answer 1

You should really use QuickCheck or similar, like hnefatl said.
But just for the fun of it, let's get your idea to work.
So you have a polymorphic function and a lot of test cases of different types. The only thing that matters is that you can apply the function each of the types.
So let's have a look at your function. It's of type [a] -> a. How should your test data look like? It should consist of a list, of a value, and it should support equality comparison. That leads you to a definition like:
{-# LANGUAGE GADTs #-}
{-# LANGUAGE ImpredicativeTypes #-}
{-# LANGUAGE RankNTypes #-}
data TestData where
TestData :: Eq a => [a] -> a -> TestData
You need to enable the GADTs language extension for this to work. For the following to work, you need these other two extensions (although the whole thing can be generalised with type classes to avoid that, just look at QuickCheck).
Now test it:
head1 = head
head2 (a : as) = a
test :: (forall a . [a] -> a) -> TestData -> Bool
test f (TestData as a) = f as == a
testAll :: [(forall a . [a] -> a)] -> [TestData] -> Bool
testAll fs testDatas = and $ test <$> fs <*> testDatas
main = putStrLn $ if testAll [head1, head2] [TestData "Foo" 'F', TestData [1..] 1]
then "Success!"
else "Oh noez!"
I'll leave it to you to generalise this for different test function types.