Consider this code which I used to solve Euler Problem 58:
diagNums = go skips 2
where go (s:skips) x = let x' = x+s
in x':go skips (x'+1)
squareDiagDeltas = go diagNums
where go xs = let (h,r) = splitAt 4 xs
in h:go r
I don't like the pattern matching in the second function. It looks more complicated than necessary! This is something that arises pretty frequently for me. Here, splitAt returns a tuple, so I have to destructure it first before I can recurse. The same pattern arises perhaps even more annoyingly when my recursion itself returns a tuple I want to modify. Consider:
f n = go [1..n]
where go [] = (0,0)
go (x:xs) = let (y,z) = go xs
in (y+x, z-x)
compared to the nice and simple recursion:
f n = go [1..n]
where go [] = 0
go (x:xs) = x+go xs
Of course the functions here are pure nonsense and could be written in a wholly different and better way. But my point is that the need for pattern matching arises every time I need to thread more than one value back through the recursion.
Are there any ways to avoid this, perhaps by using Applicative or anything similar? Or would you consider this style idiomatic?
First of all, that style is actually rather idiomatic. Since you're doing two things to two different values, there is some irreducible complexity; the actual pattern match does not introduce much on its own. Besides, I personally find the explicit style very readable most of the time.
However, there is an alternative. Control.Arrow has a bunch of functions for working with tuples. Since the function arrow -> is an Arrow as well, all these work for normal functions.
So you could rewrite your second example using (***) to combine two functions to work over tuples. This operator has the following type:
(***) :: a b c -> a b' c' -> a (b, b') (c, c')
If we replace a with ->, we get:
(***) :: (b -> c) -> (b' -> c') -> ((b, b') -> (c, c'))
So you could combine (+ x) and (- x) into a single function with (+ x) *** (- x). This would be equivalent to:
\ (a, b) -> (a + x, b - x)
Then you could use it in your recursion. Unfortunately, the - operator is stupid and doesn't work in sections, so you would have to write it with a lambda:
(+ x) *** (\ a -> a - x) $ go xs
You can obviously imagine using any other operator, all of which aren't quite as stupid :).
Honestly, I think this version is less readable than the original. However, in other cases, the *** version can be more readable, so it's useful to know about it. In particular, if you were passing (+ x) *** (- x) into a higher-order function instead of applying it immediately, I think the *** version would be better than an explicit lambda.
I agree with Tikhon Jelvis that there is nothing wrong with your version. Like he said, using combinators from Control.Arrow can be useful with higher order functions. You can write f using a fold:
f n = foldr (\x -> (+ x) *** subtract x) (0,0) [1..n]
And if you really want to get rid of the let in squareDiagDeltas (I'm not sure I would), you can use second, because you are only modifying the second element of the tuple:
squareDiagDeltas = go diagNums
where go = uncurry (:) . second go . splitAt 4
I agree with hammar, unfoldr is the way to go here.
You can also get rid of the pattern matching in diagNums:
diagNums = go skips 2
where go (s:skips) x = let x' = x+s
in x':go skips (x'+1)
The recursion makes it a little difficult to tell what's going on here, so let's
examine it in depth.
Suppose skips = s0 : s1 : s2 : s3 : ..., then we have:
diagNums = go skips 2
= go (s0 : s1 : s2 : s3 : ...) 2
= s0+2 : go (s1 : s2 : s3 : ... ) (s0+3)
= s0+2 : s0+s1+3 : go (s2 : s3 : ... ) (s0+s1+4)
= s0+2 : s0+s1+3 : s0+s1+s2+4 : go (s3 : ... ) (s0+s1+s2+5)
= s0+2 : s0+s1+3 : s0+s1+s2+4 : s0+s1+s2+s3+5 : go (...) (s0+s1+s2+s3+6)
This makes it much clearer what's going on, we've got the sum of two sequences, which is easy to compute using zipWith (+):
diagNums = zipWith (+) [2,3,4,5,...] [s0, s0+s1, s0+s1+s2, s0+s1+s2+s3,...]
So now we just need to find a better way to compute the partial sums of skips, which is a great use for scanl1:
scanl1 (+) skips = s0 : s0+s1 : s0+s1+s2 : s0+s1+s2+s3 : ...
Leaving a (IMO) much easier to understand definition for diagNums:
diagNums = zipWith (+) [2..] $ scanl1 (+) skips
Related
Consider the following situation. I define a function to process a list of elements by the typical way of doing an operation on the head and recalling the function over the rest of the list. But under certain condition of the element (being negative, being a special character, ...) I change sign on the rest of the list before continuing. Like this:
f [] = []
f (x : xs)
| x >= 0 = g x : f xs
| otherwise = h x : f (opposite xs)
opposite [] = []
opposite (y : ys) = negate y : opposite ys
As opposite (opposite xs) = xs, I become to the situation of redundant opposite operations, accumulating opposite . opposite . opposite ....
It happens with other operations instead of opposite, any such that the composition with itself is the identity, like reverse.
Is it possible to overcome this situation using functors / monads / applicatives / arrows? (I don't understand well those concepts). What I would like is to be able of defining a property, or a composition pattern, like this:
opposite . opposite = id -- or, opposite (opposite y) = y
in order that the compiler or interpreter avoids to calculate the opposite of the opposite (it is possible and simple (native) in some concatenative languages).
You can solve this without any monads, since the logic is quite simple:
f g h = go False where
go _ [] = []
go b (x':xs)
| x >= 0 = g x : go b xs
| otherwise = h x : go (not b) xs
where x = (if b then negate else id) x'
The body of the go function is almost identical to that of your original f function. The only difference is that go decides whether the element should be negated or not based on the boolean value passed to it from previous calls.
Sure, just keep a bit of state telling whether to apply negate to the current element or not. Thus:
f = mapM $ \x_ -> do
x <- gets (\b -> if b then x_ else negate x_)
if x >= 0
then return (g x)
else modify not >> return (h x)
I found this statement while studying Functional Reactive Programming, from "Plugging a Space Leak with an Arrow" by Hai Liu and Paul Hudak ( page 5) :
Suppose we wish to define a function that repeats its argument indefinitely:
repeat x = x : repeat x
or, in lambdas:
repeat = λx → x : repeat x
This requires O(n) space. But we can achieve O(1) space by writing instead:
repeat = λx → let xs = x : xs
in xs
The difference here seems small but it hugely prompts the space efficiency. Why and how it happens ? The best guess I've made is to evaluate them by hand:
r = \x -> x: r x
r 3
-> 3: r 3
-> 3: 3: 3: ........
-> [3,3,3,......]
As above, we will need to create infinite new thunks for these recursion. Then I try to evaluate the second one:
r = \x -> let xs = x:xs in xs
r 3
-> let xs = 3:xs in xs
-> xs, according to the definition above:
-> 3:xs, where xs = 3:xs
-> 3:xs:xs, where xs = 3:xs
In the second form the xs appears and can be shared between every places it occurring, so I guess that's why we can only require O(1) spaces rather than O(n). But I'm not sure whether I'm right or not.
BTW: The keyword "shared" comes from the same paper's page 4:
The problem here is that the standard call-by-need evaluation rules
are unable to recognize that the function:
f = λdt → integralC (1 + dt) (f dt)
is the same as:
f = λdt → let x = integralC (1 + dt) x in x
The former definition causes work to be repeated in the recursive call
to f, whereas in the latter case the computation is shared.
It's easiest to understand with pictures:
The first version
repeat x = x : repeat x
creates a chain of (:) constructors ending in a thunk which will replace itself with more constructors as you demand them. Thus, O(n) space.
The second version
repeat x = let xs = x : xs in xs
uses let to "tie the knot", creating a single (:) constructor which refers to itself.
Put simply, variables are shared, but function applications are not. In
repeat x = x : repeat x
it is a coincidence (from the language's perspective) that the (co)recursive call to repeat is with the same argument. So, without additional optimization (which is called static argument transformation), the function will be called again and again.
But when you write
repeat x = let xs = x : xs in xs
there are no recursive function calls. You take an x, and construct a cyclic value xs using it. All sharing is explicit.
If you want to understand it more formally, you need to familiarize yourself with the semantics of lazy evaluation, such as A Natural Semantics for Lazy Evaluation.
Your intuition about xs being shared is correct. To restate the author's example in terms of repeat, instead of integral, when you write:
repeat x = x : repeat x
the language does not recognize that the repeat x on the right is the same as the value produced by the expression x : repeat x. Whereas if you write
repeat x = let xs = x : xs in xs
you're explicitly creating a structure that when evaluated looks like this:
{hd: x, tl:|}
^ |
\________/
In explaining foldr to Haskell newbies, the canonical definition is
foldr :: (a -> b -> b) -> b -> [a] -> b
foldr _ z [] = z
foldr f z (x:xs) = f x (foldr f z xs)
But in GHC.Base, foldr is defined as
foldr k z = go
where
go [] = z
go (y:ys) = y `k` go ys
It seems this definition is an optimization for speed, but I don't see why using the helper function go would make it faster. The source comments (see here) mention inlining, but I also don't see how this definition would improve inlining.
I can add some important details about GHC's optimization system.
The naive definition of foldr passes around a function. There's an inherent overhead in calling a function - especially when the function isn't known at compile time. It'd be really nice to able to inline the definition of the function if it's known at compile time.
There are tricks available to perform that inlining in GHC - and this is an example of them. First, foldr needs to be inlined (I'll get to why later). foldr's naive implementation is recursive, so cannot be inlined. So a worker/wrapper transformation is applied to the definition. The worker is recursive, but the wrapper is not. This allows foldr to be inlined, despite the recursion over the structure of the list.
When foldr is inlined, it creates a copy of all of its local bindings, too. It's more or less a direct textual inlining (modulo some renaming, and happening after the desugaring pass). This is where things get interesting. go is a local binding, and the optimizer gets to look inside it. It notices that it calls a function in the local scope, which it names k. GHC will often remove the k variable entirely, and will just replace it with the expression k reduces to. And then afterwards, if the function application is amenable to inlining, it can be inlined at this time - removing the overhead of calling a first-class function entirely.
Let's look at a simple, concrete example. This program will echo a line of input with all trailing 'x' characters removed:
dropR :: Char -> String -> String
dropR x r = if x == 'x' && null r then "" else x : r
main :: IO ()
main = do
s <- getLine
putStrLn $ foldr dropR "" s
First, the optimizer will inline foldr's definition and simplify, resulting in code that looks something like this:
main :: IO ()
main = do
s <- getLine
-- I'm changing the where clause to a let expression for the sake of readability
putStrLn $ let { go [] = ""; go (x:xs) = dropR x (go xs) } in go s
And that's the thing the worker-wrapper transformation allows.. I'm going to skip the remaining steps, but it should be obvious that GHC can now inline the definition of dropR, eliminating the function call overhead. This is where the big performance win comes from.
GHC cannot inline recursive functions, so
foldr :: (a -> b -> b) -> b -> [a] -> b
foldr _ z [] = z
foldr f z (x:xs) = f x (foldr f z xs)
cannot be inlined. But
foldr k z = go
where
go [] = z
go (y:ys) = y `k` go ys
is not a recursive function. It is a non-recursive function with a local recursive definition!
This means that, as #bheklilr writes, in map (foldr (+) 0) the foldr can be inlined and hence f and z replaced by (+) and 0 in the new go, and great things can happen, such as unboxing of the intermediate value.
As the comments say:
-- Inline only in the final stage, after the foldr/cons rule has had a chance
-- Also note that we inline it when it has *two* parameters, which are the
-- ones we are keen about specialising!
In particular, note the "we inline it when it has two parameters, which are the ones we are keen about specialising!"
What this is saying is that when foldr gets inlined, it's getting inlined only for the specific choice of f and z, not for the choice of the list getting folded. I'm not expert, but it would seem it would make it possible to inline it in situations like
map (foldr (+) 0) some_list
so that the inline happens in this line and not after map has been applied. This makes it optimizable in more situations and more easily. All the helper function does is mask the 3rd argument so {-# INLINE #-} can do its thing.
One tiny important detail not mentioned in other answers is that GHC, given a function definition like
f x y z w q = ...
cannot inline f until all of the arguments x, y, z, w, and q are applied. This means that it's often advantageous to use the worker/wrapper transformation to expose a minimal set of function arguments which must be applied before inlining can occur.
I was trying to implement a Haskell function that takes as input an array of integers A
and produces another array B = [A[0], A[0]+A[1], A[0]+A[1]+A[2] ,... ]. I know that scanl from Data.List can be used for this with the function (+). I wrote the second implementation
(which performs faster) after seeing the source code of scanl. I want to know why the first implementation is slower compared to the second one, despite being tail-recursive?
-- This function works slow.
ps s x [] = x
ps s x y = ps s' x' y'
where
s' = s + head y
x' = x ++ [s']
y' = tail y
-- This function works fast.
ps' s [] = []
ps' s y = [s'] ++ (ps' s' y')
where
s' = s + head y
y' = tail y
Some details about the above code:
Implementation 1 : It should be called as
ps 0 [] a
where 'a' is your array.
Implementation 2: It should be called as
ps' 0 a
where 'a' is your array.
You are changing the way that ++ associates. In your first function you are computing ((([a0] ++ [a1]) ++ [a2]) ++ ...) whereas in the second function you are computing [a0] ++ ([a1] ++ ([a2] ++ ..)). Appending a few elements to the start of the list is O(1), whereas appending a few elements to the end of a list is O(n) in the length of the list. This leads to a linear versus quadratic algorithm overall.
You can fix the first example by building the list up in reverse order, and then reversing again at the end, or by using something like dlist. However the second will still be better for most purposes. While tail calls do exist and can be important in Haskell, if you are familiar with a strict functional language like Scheme or ML your intuition about how and when to use them is completely wrong.
The second example is better, in large part, because it's incremental; it immediately starts returning data that the consumer might be interested in. If you just fixed the first example using the double-reverse or dlist tricks, your function will traverse the entire list before it returns anything at all.
I would like to mention that your function can be more easily expressed as
drop 1 . scanl (+) 0
Usually, it is a good idea to use predefined combinators like scanl in favour of writing your own recursion schemes; it improves readability and makes it less likely that you needlessly squander performance.
However, in this case, both my scanl version and your original ps and ps' can sometimes lead to stack overflows due to lazy evaluation: Haskell does not necessarily immediately evaluate the additions (depends on strictness analysis).
One case where you can see this is if you do last (ps' 0 [1..100000000]). That leads to a stack overflow. You can solve that problem by forcing Haskell to evaluate the additions immediately, for instance by defining your own, strict scanl:
myscanl :: (b -> a -> b) -> b -> [a] -> [b]
myscanl f q [] = []
myscanl f q (x:xs) = q `seq` let q' = f q x in q' : myscanl f q' xs
ps' = myscanl (+) 0
Then, calling last (ps' [1..100000000]) works.
I am reading a tutorial that uses the following example (that I'll generalize somewhat):
f :: Foo -> (Int, Foo)
...
fList :: Foo -> [Int]
fList foo = x : fList bar
where
(x, bar) = f foo
My question lies in the fact that it seems you can refer to x and bar, by name, outside of the tuple where they are obtained. This would seem to act like destructuring parameter lists in other languages, if my guess is correct. (In other words, I didn't have to do the following:)
fList foo = (fst tuple) : fList (snd tuple)
where
tuple = f foo
Am I right about this behavior? I've never seen it mentioned yet in the tutorials/books I've been reading. Can someone point me to more info on the subject?
Edit: Can anything (lists, arrays, etc.) be destructured in a similar way, or can you only do this with tuples?
Seeing your edit, I think what your asking about is Pattern matching.
And to answer your question: Yes, anything you can construct, you can also 'deconstruct' using the constructors. For example, you're probably familiar with this form of pattern matching:
head :: [a] -> a
head (x:xs) = x
head [] = error "Can't take head of empty list"
However, there are more places where you can use pattern matching, other valid notations are:
head xs = case xs of
(y:ys) -> y
[] -> error "Can't take head of empty list"
head xs = let (y:ys) = xs
in y
head xs = y
where
(y:ys) = xs
Note that the last two examples are a bit different from the first to because they give different error messages when you call them with an empty list.
Although these examples are specific to lists, you can do the same with other data types, like so:
first :: (a, b) -> a
first tuple = x
where
(x, y) = tuple
second :: (a, b) -> b
second tuple = let (x, y) = tuple
in y
fromJust :: Maybe a -> a
fromJust ma = x
where
(Just x) = ma
Again, the last function will also crash if you call it with Nothing.
To sum up; if you can create something using constructors (like (:) and [] for lists, or (,) for tuples, or Nothing and Just for Maybe), you can use those same constructors to do pattern matching in a variety of ways.
Am I right about this behavior?
Yes. The names exist only in the block where you have defined them, though. In your case, this means the logical unit that your where clause is applied to, i.e. the expression inside fList.
Another way to look at it is that code like this
x where x = 3
is roughly equivalent to
let x = 3 in x
Yes, you're right. Names bound in a where clause are visible to the full declaration preceding the where clause. In your case those names are f and bar.
(One of the hard things about learning Haskell is that it is not just permitted but common to use variables in the source code in locations that precede the locations where those variables are defined.)
The place to read more about where clauses is in the Haskell 98 Report or in one of the many fine tutorials to be found at haskell.org.