Using QuickCheck to test intentional error conditions - haskell

I've seen how QuickCheck can be used to test monadic and non-monadic code, but how can I use it to test code that handles errors, i.e., prints some message and then calls exitWith?

A disclaimer first: I'm not an expert on QuickCheck and I had no experience with monadic checking before your question, but I see stackoverflow as an opportunity to learn new things. If there's an expert answer saying this can be done better, I'll remove mine.
Say you have a function test that can throw exceptions using exitWith. Here's how I think you can test it. The key function is protect, which catches the exception and converts it to something you can test against.
import System.Exit
import Test.QuickCheck
import Test.QuickCheck.Property
import Test.QuickCheck.Monadic
test :: Int -> IO Int
test n | n > 100 = do exitWith $ ExitFailure 1
| otherwise = do print n
return n
purifyException :: (a -> IO b) -> a -> IO (Maybe b)
purifyException f x = protect (const Nothing) $ return . Just =<< f x
testProp :: Property
testProp = monadicIO $ do
input <- pick arbitrary
result <- run $ purifyException test $ input
assert $ if input <= 100 then result == Just input
else result == Nothing
There are two disadvantages to this, as far as I can see, but I found no way over them.
I found no way to extract the ExitCode exception from the AnException that protect can handle. Therefore, all exit codes are treated the same here (they are mapped to Nothing). I would have liked to have:
purifyException :: (a -> IO b) -> a -> IO (Either a ExitCode)
I found no way to test the I/O behavior of test. Suppose test was:
test :: IO ()
test = do
n <- readLn
if n > 100 then exitWith $ ExitFailure 1
else print n
Then how would you test it?
I'd appreciate more expert answers too.

The QuickCheck expectFailure function can be used to handle this type of thing. Take this simple (and not-recommended) error-handling framework:
import System.Exit
import Test.QuickCheck
import Test.QuickCheck.Monadic
handle :: Either a b -> IO b
handle (Left _) = putStrLn "exception!" >> exitWith (ExitFailure 1)
handle (Right x) = return x
and whip up a couple of dummy functions:
positive :: Int -> Either String Int
positive x | x > 0 = Right x
| otherwise = Left "not positive"
negative :: Int -> Either String Int
negative x | x < 0 = Right x
| otherwise = Left "not negative"
Now we can test some properties of the error handling. First, Right values should not result in exceptions:
prop_returnsHandledProperly (Positive x) = monadicIO $ do
noErr <- run $ handle (positive x)
assert $ noErr == x
-- Main*> quickCheck prop_returnsHandledProperly
-- +++ OK, passed 100 tests.
Lefts should result in exceptions. Notice the expectFailure tacked on to the start:
prop_handlesExitProperly (Positive x) = expectFailure . monadicIO $
run $ handle (negative x)
-- Main*> quickCheck prop_handlesExitProperly
-- +++ OK, failed as expected. Exception: 'exitWith: invalid argument (ExitFailure 0)' (after 1 test):

Related

Technique for reading in multiple lines for Haskell IO

Basically I would like to find a way so that a user can enter the number of test cases and then input their test cases. The program can then run those test cases and print out the results in the order that the test cases appear.
So basically I have main which reads in the number of test cases and inputs it into a function that will read from IO that many times. It looks like this:
main = getLine >>= \tst -> w (read :: String -> Int) tst [[]]
This is the method signature of w: w :: Int -> [[Int]]-> IO ()
So my plan is to read in the number of test cases and have w run a function which takes in each test case and store the result into the [[]] variable. So each list in the list will be an output. w will just run recursively until it reaches 0 and print out each list on a separate line. I'd like to know if there is a better way of doing this since I have to pass in an empty list into w, which seems extraneous.
As #bheklilr mentioned you can't update a value like [[]]. The standard functional approach is to pass an accumulator through a a set of recursive calls. In the following example the acc parameter to the loop function is this accumulator - it consists of all of the output collected so far. At the end of the loop we return it.
myTest :: Int -> [String]
myTest n = [ "output line " ++ show k ++ " for n = " ++ show n | k <- [1..n] ]
main = do
putStr "Enter number of test cases: "
ntests <- fmap read getLine :: IO Int
let loop k acc | k > ntests = return $ reverse acc
loop k acc = do
-- we're on the kth-iteration
putStr $ "Enter parameter for test case " ++ show k ++ ": "
a <- fmap read getLine :: IO Int
let output = myTest a -- run the test
loop (k+1) (output:acc)
allOutput <- loop 1 []
print allOutput
As you get more comfortable with this kind of pattern you'll recognize it as a fold (indeed a monadic fold since we're doing IO) and you can implement it with foldM.
Update: To help explain how fmap works, here are equivalent expressions written without using fmap:
With fmap: Without fmap:
n <- fmap read getLine :: IO [Int] line <- getLine
let n = read line :: Int
vals <- fmap (map read . words) getLine line <- getLine
:: IO [Int] let vals = (map read . words) line :: [Int]
Using fmap allows us to eliminate the intermediate variable line which we never reference again anyway. We still need to provide a type signature so read knows what to do.
The idiomatic way is to use replicateM:
runAllTests :: [[Int]] -> IO ()
runAllTests = {- ... -}
main = do
numTests <- readLn
tests <- replicateM numTests readLn
runAllTests tests
-- or:
-- main = readLn >>= flip replicateM readLn >>= runAllTests

How to get output of function returning Maybe?

This is a newbie question.
I am having a trouble understanding the output of StripPrefix function which returns Maybe [a].
What I am doing is, I am passing two strings to StripPrefix so that it gives back the string after cutting the prefix.
What I have tried is :
let b = stripPrefix pref stri
Just b <- stripPrefix pref stri
In first case, my print operation (putStrLn b) throws error "Couldn't match type Maybe [Char]' with[Char]'"
From the comments on the question:
In GHCi, if you want to extract the a from a Maybe a you have a few options. First, if you're sure it will succeed with a Just something, you can do
> let Just a = Just 1
> print a
1
However, this can lead to problems if your operation is not successful
> let Just a = Nothing :: Maybe Int
> print a
*** Exception <interactive>12:5-20: Irrefutable pattern failed for pattern Data.Maybe.Just a
All this is saying is that the pattern matching you used failed. How do we avoid this? There's case statements:
> -- Enable multiline input (I have this in my .ghci file so it's always on)
> :set +m
> let maybeA = Just 1
|
> case maybeA of
| Just x -> print x
| Nothing -> return () -- Do nothing
|
1
But this is laborious. Wouldn't it be nice if there was an alternative built-in to Haskell? Fortunately, there is in the Data.Maybe module:
> import Data.Maybe
> :type maybe
maybe :: b -> (a -> b) -> Maybe a -> b
> -- The -1 is our default value in case of Nothing
> print $ maybe (-1) id $ Just 1
1
> print $ maybe (-1) id $ Nothing
-1
There's even an easier function to use when all you want is either the value in a Just or a default value:
> print $ fromMaybe (-1) $ Just 1
1
But maybe is more powerful in general:
> print $ maybe 0 (\x -> 2 * x - x * x * x + 7 ^ x) $ Just 3
322
There are still times, though, that all you want to know is if an operation was successful. For that, Data.Maybe has isJust and isNothing:
> isJust $ Just 1
True
> isJust $ Nothing
False
And isNothing = not . isJust, obviously.
That's because putStrLn :: String -> IO () and b :: Maybe String. putStrLn expects its first argument to be a String, and that's not what b is. You can use print :: Show a => a -> IO () to print a Maybe value, provided that the type it contains is itself Showable.

How to monitor computation process in Haskell

I have a function in my main block
map anyHeavyFunction [list]
I'd like to show a progress bar during the computation process or add additional actions (pause, stop process etc.), but because map is a pure function I can't do it directly. I can guess I have to use monads, but what monad is appropriate? IO, State?
I know there is at least one library on hackage that has some pre-made monad transformers for this task, but I normally turn to the pipes package to roll my own when I need one. I am using pipes-4.0.0 it is going to be on hackage this weekend, but you can grab it form the github repo before that.
I also used terminal-progress-bar package so that it makes a nice terminal animation as well.
{-# language BangPatterns #-}
import Pipes
import qualified Pipes.Prelude as P
import Control.Monad.IO.Class
import System.ProgressBar
import System.IO ( hSetBuffering, BufferMode(NoBuffering), stdout )
-- | Takes the total size of the stream to be processed as l and the function
-- to map as fn
progress l = loop 0
where
loop n = do
liftIO $ progressBar (msg "Working") percentage 40 n l
!x <- await -- bang pattern to make strict
yield x
loop (n+1)
main = do
-- Force progress bar to print immediately
hSetBuffering stdout NoBuffering
let n = 10^6
let heavy x = last . replicate n $ x -- time wasting function
r <- P.toListM $ each [1..100] >-> P.map heavy >-> progress 100
putStrLn ""
return r
This animates:
> Working [=>.......................] 7%
> Working [=====>...................] 20%
Every update erases the last bar so it only take up one line on the terminal. Then it finishes like so:
> main
Working [=========================] 100%
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100]
Here's a (kind of) simple answer that I'm not satisfied with. It is based on the fact that #shellenberg wanted to apply a heavy function on each element of a (supposedly long) list. If it suffices to move the "progress bar" once for every element of the list, then the following can be turned into a general solution.
First of all, you need to pick the monad in which you'll work. This depends on what exactly your "progress bar" is. For this discussion, let's say that the IO monad is enough and that we want to alternately display the characters -, /, | and \. You'll also (most probably) need some kind of state S (here it is only the number of elements processed so far, therefore S is Int), so the real monad used will be StateT S IO.
Suppose your original program is:
m = 100000 -- how many elements the list has
-- Your (pure) function
anyHeavyFunction :: Int -> Bool
anyHeavyFunction n =
length [1..n] + length [n+1..4217] == 4217
-- Your list
list :: [Int]
list = take m $ repeat 4217
-- The main program
main :: IO ()
main = do
let l = map anyHeavyFunction list
if and l
then putStrLn "OK"
else putStrLn "WRONG"
(Notice that, very conveniently, the heavy function takes the same time for each element of the list.)
This is how you could convert it to display the crude "progress bar":
import Control.Monad.State
import System.IO (hFlush, stdout)
m = 100000 -- how many elements the list has
k = 5000 -- how often you want to "tick"
tick :: a -> StateT Int IO a
tick x = do
s <- get
put $ s+1
when (s `mod` k == 0) $ liftIO $ do
let r = (s `div` k) `mod` 4
putChar $ "-/|\\" !! r
putChar '\b'
hFlush stdout
x `seq` return x
-- Your (pure) function
anyHeavyFunction :: Int -> Bool
anyHeavyFunction n =
length [1..n] + length [n+1..4217] == 4217
-- Your list
list :: [Int]
list = take m $ repeat 4217
-- The main program
main :: IO ()
main = do
l <- flip evalStateT 0 $ mapM (tick . anyHeavyFunction) list
if and l
then putStrLn "OK"
else putStrLn "WRONG"
An interesting point: The seq in tick forces evaluation of the result for each element of the list. This is enough, if the result has a basic type (Bool here). Otherwise, it's not clear what you would want to do -- remember Haskell is lazy!
If one wants a finer progress bar or if one is not satisfied with the assumption that one "tick" will be counted for each element of the list, then I believe it's necessary to incorporate the ticking in the logic of the heavy function. This makes it ugly... I'd like to see what kind of general solutions can be suggested to that. I'm all in for Haskell, but I think it just sucks for such things as progress bars... There's no free lunch; you can't be pure and lazy and have your progress bars made easy!
EDIT: A version which uses the ProgressBar module suggested by #Davorak. It certainly looks nicer than my rotating bar.
import Control.Monad.State
import System.ProgressBar
import System.IO (hSetBuffering, BufferMode(NoBuffering), stdout)
m = 100000 -- how many elements the list has
k = 5000 -- how often you want to "tick"
tick :: a -> StateT Int IO a
tick x = do
s <- get
put $ s+1
when (s `mod` k == 0) $ liftIO $ do
progressBar (msg "Working") percentage 40 (toInteger s) (toInteger m)
x `seq` return x
-- Your (pure) function
anyHeavyFunction :: Int -> Bool
anyHeavyFunction n =
length [1..n] + length [n+1..4217] == 4217
-- Your list
list :: [Int]
list = take m $ repeat 4217
-- The main program
main :: IO ()
main = do
hSetBuffering stdout NoBuffering
l <- flip evalStateT 0 $ mapM (tick . anyHeavyFunction) list
if and l
then putStrLn "OK"
else putStrLn "WRONG"
The idea is the same, the drawbacks too.
You could use parMap to apply the expensive function in parallel (if the dependencies permit) and a list of TVars corresponding to each list (or chunk of) element(s) and set them once the respective function application has completed. A separate thread could check on the values and update the display (obviously some IO action would happen here).

Haskell quickcheck issue

I am writing a simple test using quickcheck.
import Test.QuickCheck
f :: Int -> Int
f x
| x < 0 = (-x)
| otherwise = x
main = do
putStrLn "Testing"
quickCheck ((\x -> ((f x) >= 0)) :: Int -> Bool)
Whenever I run this via $ runhaskell test.hs, I see one of 2 different results.
Either I get:
Testing
+++ OK, passed 100 tests.
Or:
I get no output, and the program terminates.
I cannot reason about this behavior.
This is on Quickcheck 2.5.1.1, and ghc 7.4.2.

How to count the number of times a function was called, the FP way

I am currently working through SICP with Haskell. Exercise 1.15 asks how many times a function is called. The idea is probably that you should use the substitution method, but I would like to know how to do so in code.
In an imperative language one can keep a global variable and increment it every time the function is called. But how would you go about it in Haskell (or the pure functional way)?
You can use the Writer monad to accomplish this, provided that all of the calls to the function in question can be grouped together into a do block:
import Control.Monad.Writer
myFunc :: Int -> Int -> Writer (Sum Int) Int
myFunc a b = tell (Sum 1) >> return (a + b)
callMyFunc :: ((Int, Int, Int), Sum Int)
callMyFunc = runWriter $ do a <- myFunc 2 3
b <- myFunc 8 7
c <- myFunc 3 5
return (a, b, c)
main = putStrLn $
"myFunc was called "
++ show (getSum $ snd callMyFunc)
++ " times and produced "
++ show (fst callMyFunc)
Which outputs:
myFunc was called 3 times and produced (5,15,8)
It sounds to me like you need to have some kind of counter regardless of whether you go with a functional or a non-functional way. In Haskell, you could use the State Monad to keep track of the state:
import Control.Monad.State
someFunc x = do
num <- get
put (num + 1)
return $ x * x
runSomeFuncs = do
someFunc 1
someFunc 2
someFunc 3
main = do
let (res, state) = runState runSomeFuncs 0
putStrLn ("result: " ++ (show res))
putStrLn ("# of calls: " ++ show state)
Here, you want to keep track of how many times someFunc got called, so we pass an integer in as the state and increment the integer every time the function gets called by using:
num <- get
put (num + 1)
and then increment it by 1 and put it back. If you run this script, it should print
result: 9
# of calls: 3

Resources