The Foldr Foldl Foldl' wiki page describes the differences between foldr and foldl. Both process lists from left-to-right, but foldr accumulates the result from right-to-left whereas foldl does so from left-to-right.
The page goes on to discourage the use of foldl in favor of the eager version foldl', which is more efficient.
Is there a corresponding eager version of foldr, presumably called foldr'? If so, is there a reason that it isn't mentioned on the wiki page?
There's no need for foldr', sonce one can always use foldr f with a strict f to achieve the same goal.
We could define it...
foldr' f a xs = foldr f' a xs
where f' x y = seq x (seq y (f x y))
... but it's simpler to pass a strict f at the call point instead, if needed.
A quick Hoogle search shows that there is, indeed, a foldr' defined in Data.Foldable.
I assume the reason it isn't mentioned on the linked page is that it isn't relevant to the problem of stack overflows discussed there. When calculating a sum using foldr (+) 0 on a large list, the stack overflow that may occur isn't a result of lazy application of the operator (+). Rather, it occurs because a fundamental feature of a right fold (whether strict or lazy) is that the most deeply nested application of the operator occurs at the end of the list. For an operator like (+) that requires evaluation of both operands to yield any sort of result, this means that either foldr or foldr' must build up an O(n) stack of continuations before getting to the end of the list where the (+) operator can start to do "real work".
Both foldr and foldl' are extremely useful for lists. Neither foldl nor foldr' is often useful for lists. But these days, those functions are actually methods of the Foldable class, and they are useful for some instances. To take an extreme example, consider snoc lists:
data SL a = SL a :> a | Nil
infixl 5 :>
instance Foldable SL where
foldMap _ Nil = mempty
foldMap f (xs :> x) = foldMap f xs <> f x
foldl _ b Nil = b
foldl f b (xs :> x) = f (foldl f b xs) x
foldr' _ n Nil = n
foldr' c n (xs :> x) =
let n' = c x n
in n' `seq` foldr' c n' xs
For snoc lists, foldl and foldr' are really useful, while foldr and foldl' are pretty much useless!
Many Foldable containers can make good use of all four of these methods. For example, Data.Set.Set, Data.Map.Map k and Data.Sequence.Seq can each be folded perfectly well in either direction.
import Data.Sequence (Seq)
import qualified Data.Sequence as S
toForwardList :: Seq a -> [a]
toForwardList = foldr (:) []
toReverseList :: Seq a -> [a]
toReverseList = foldl (flip (:)) []
The forward and reverse conversions are equally efficient and equally lazy.
Related
In the same scenario as my previous question, I was trying to implement the cycle function only¹ using a fold, when I came up with the following wrong function, which tries to concatenate the accumulator with itself, building the infinite list exponentially (yes, I know this would mean that it generates 2048 copies if one wants to take 1025 of them)
myCycle :: [a] -> [a]
myCycle s = foldr (\_ a -> a ++ a) s [1..]
However, using it throws *** Exception: heap overflow.
This version, instead, works like a charm
myCycle :: [a] -> [a]
myCycle s = foldr (\_ a -> s ++ a) s [1..]
My question is, why does the former version overflow, as compared to the latter? I feel the reason is dumber than me...
[1] I mean, implementing cycle as a fold, having only the step function and the seed as degrees of freedom.
foldr c n takes a list and replaces every (:) with c and the final [] with n. But [1..] has no final [], so foldr (\_ a -> a ++ a) s has no place to put the s. Therefore no information ever "flows" from s to the result of myCycle s, which means it has no choice but to be bottom (or rather: it has too much choice—it's underspecified—so it gives up and falls back to bottom). The second version actually does use s, because it appears in the folding function, which is used when foldr acts on an infinite list.
In fact, we have the identity
foldr (\_ -> f) x xs = fix f = let x = f x in x
when xs is infinite. That is, the second argument of foldr is completely ignored when the list is infinite. Further, if that folding function doesn't actually look at the elements of the list, then all that's really happening is you're infinitely nesting f within itself: fix f = f (f (f (f ...))). fix is fundamental in the sense that every kind of recursion can be written in terms of it (certain more exotic kinds of recursion require adding some language extensions, but the definition fix f = let x = f x in x itself doesn't change). This makes writing any recursive function in terms of foldr and an infinite list trivial.
Here's my take on an exponential cycle. It produces 1 copy of the input, concatenated onto 2 copies, concatenated onto 4, etc.
myCycle xs = xs ++ myCycle (xs ++ xs)
You translate an explicitly recursive definition to fix by abstracting the recursive call as a parameter and passing that to fix:
myCycle = fix \rec xs -> xs ++ rec (xs ++ xs)
And then you use the foldr identity and introduce a bogus [] case
myCycle = foldr (\_ rec xs -> xs ++ rec (xs ++ xs)) (error "impossible") [1..]
I am learning Arrow following the tutorial programming with arrows. I've typed the following code according to the paper except that the SF is defined by data, not by newtype as in the paper (actually, I made this change by chance, since I typed the code from memory):
import Control.Category
import Control.Arrow
import Prelude hiding (id, (.))
data SF a b = SF { runSF :: [a] -> [b] } -- this is the change, using data instead of newtype as in the paper
-- The folowing code is the same as in the paper
instance Category SF where
id = SF $ \x -> x
(SF f) . (SF g) = SF $ \x -> f (g x)
instance Arrow SF where
arr f = SF $ map f
first (SF f) = SF $ unzip >>> first f >>> uncurry zip
instance ArrowChoice SF where
left (SF f) = SF $ \xs -> combine xs (f [y | Left y <- xs])
where
combine (Left _ : ys) (z:zs) = Left z : combine ys zs
combine (Right y : ys) zs = Right y : combine ys zs
combine [] _ = []
delay :: a -> SF a a
delay x = SF $ init . (x:)
mapA :: ArrowChoice a => a b c -> a [b] [c]
mapA f = arr listcase >>>
arr (const []) ||| (f *** mapA f >>> arr (uncurry (:)))
listcase :: [a] -> Either () (a, [a])
listcase [] = Left ()
listcase (x:xs) = Right (x, xs)
When I load the file in ghci and execute runSF (mapA (delay 0)) [[1,2,3],[4,5,6]], it triggers an infinit loop and runs out of memory finally. If I change data back to newtype, everything is OK. The same problem happens in ghc 8.0.2, 8.2.2 and 8.6.3.
The same problem also exists even I compile the code into an executable.
I have thought the difference between data and newtype, when defining a data structure with only one field, is the runtime cost. But this problem seems to imply more difference between them. Or there may be something that I haven't noticed about the Arrow type-class.
Can anyone have any ideas? Thanks very much!
Let's look at this example.
data A = A [Int]
deriving (Show)
cons :: Int -> A -> A
cons x (A xs) = A (x:xs)
ones :: A
ones = cons 1 ones
We would expect that ones should be A [1,1,1,1...], because all we have done is wrap a list in a data constructor. But we would be wrong. Recall that pattern matches are strict for data constructors. That is, cons 1 undefined = undefined rather than A (1 : undefined). So when we try to evaluate ones, cons pattern matches on its second argument, which causes us to evaluate ones... we have a problem.
newtypes don't do this. At runtime newtype constructors are invisible, so it's as if we had written the equivalent program on plain lists
cons :: Int -> [Int] -> [Int]
cons x ys = x:ys
ones = cons 1 ones
which is perfectly productive, since when we try to evaluate ones, there is a : constructor between us and the next evaluation of ones.
You can get back the newtype semantics by making your data constructor pattern matches lazy:
cons x ~(A xs) = A (x:xs)
This is the problem with your code (I have run into this exact problem doing this exact thing). There are a few reasons data pattern matches are strict by default; the most compelling one I see is that pattern matching would otherwise be impossible if the type had more than one constructor. There is also a small runtime overhead to lazy pattern matching in order to fix some subtle GC leaks; details linked in the comments.
After reading this question: Functional proofs (Haskell)
And after looking at the inductive proof of forall xs ys. length (xs ++ ys) = length xs + length ys from the Haskell School of Music (page 164).
It seemed to me that function application distributes over list concatenation.
Hence the more general law might be that forall f xs ys. f (xs ++ ys) = f xs ++ f ys.
But how would one prove/disprove such a predicate?
-- EDIT --
I made a typo it was meant to be: forall f xs ys. f (xs ++ ys) = f xs + f ys, which matches what the previous question and the Haskell SoM uses. That being said, because of this typo, it's no longer "distributivity" property. However, #leftaroundabout made the correct answer for my original typoed question. And as for my intended question, the law is still not correct, because functions don't need the preserve the structural value. The f might give a completely different answer depending on the length of the list it is applied to.
No, this is clearly not true in general:
f [_] = []
f l = l
then
f ([1] ++ [2]) = f [1,2] = [1,2]
but
f [1] ++ f [2] = [] ++ [] = []
I'm sure the functions which do have this problem form an interesting class, but general functions can do pretty much anything to a list's structure which thwarts such invariants.
And after looking at the inductive proof of forall xs ys. length (xs ++ ys) = length xs + length ys from the Haskell School of Music (page 164).
It seemed to me that function application distributes over list concatenation.
Well, clearly that is not the case. For example:
reverse ([1..3] ++ [4..6]) /= reverse [1..3] ++ reverse [4..6]
The example that you're quoting is a special case that's called a monoid morphism: a function f :: m -> n such that:
m and n are monoids with binary operation <> and identity mempty;
f mempty = mempty
f (m <> m') == f m <> f m'
So length :: [a] -> Int is a monoid morphism, sending [] to 0 and ++ to +:
length [] = 0
length (xs ++ ys) = length xs + length ys
I am using the Control.Foldl library to traverse an arbitrarily long list and counting all occurrences of arbitrarily many unique entities. Ie, the list may be of form
[Just "a", Just "b", Just "aab", Nothing, Just "aab"]
and I my result should something like:
[(Just "a",1),(Just "b",1) (Just "aab", 2), (Nothing, 1)]
Now the issue is I do not have the name of these entities a priori, and I would like to dynamically update the results as I fold.
My problem is that I do not know how to describe this computation in terms of the Fold data type from Control.foldl. Specifically, at each step of the fold I need to traverse the result list and ask if I have seen the current item, but I see no way of describing this using foldl.
Please note for future use purposes it's really important that I use the Control.Foldl library here, not fold over some other foldable data type like a map. In some sense my question is more along the lines of how to use the Foldl library, since the documentation is not too clear to me.
Edit: The example I showed is just a toy example, in reality I need to traverse a arb large list many times computing statistics, hence I'm using the foldl library, which allow me to combine the computations using applicatives ie toResults <$> stat1 <*> stat2 <*> ... <*> statm $ largeList and foldl allow me to traverse the list just once, computing all m statistics. Please find a solution using the foldl library.
You can encode a normal foldl' pretty straightforwardly as a Fold:
foldlToFold :: (b -> a -> b) -> b -> Fold a b
foldlToFold f z = Fold f z id
I'm actually a bit puzzled that this combinator isn't in the library...
Anyways, if you have
foldl' f z
you can replace it with
fold (Fold f z id)
so here, you would normally be using
foldl' (\mp x -> M.insertWith (+) x 1 mp) M.empty
with Fold, you'd be making
countingFold :: Ord a => Fold a (Map a Int)
countingFold = Fold (\mp x -> M.insertWith (+) 1 mp) M.empty id
and you can use it as
countUp :: Ord a => [a] -> Map a Int
countUp = fold countingFold
-- or
countUp = fold (Fold (\mp x -> M.insertWith (+) 1 mp) M.empty id)
If you want to go back to a list at the end, you can do
M.toList . countUp
In general, if you can formulate your fold as a foldl', you can do the transformation above to be able to encode it as a Fold. Fold is a bit more expressive because for foldl', the b type is both the accumulator and the result type; for a Fold, you can have a separate accumulator and result type.
Roughly speaking, you can translate any Fold into a foldl-and-map:
Fold f z g = map g . foldl' f z
And you can go backwards too:
foldlMapToFold :: (b -> a -> b) -> b -> (b -> c) -> Fold a c
foldlMapToFold = Fold
So if you had
map g . foldl' f z
you can write
fold (Fold f z g)
If you want to use a Fold, think, "how can i describe my operation as a foldl' and a map?", and then go from there.
The advantage of using the Fold type over just normal maps and folds is (apart from performance tweaks) the ability to combine and manipulate multiple Folds as objects using their Applicative instance, and other nice instances too, like Functor, Profunctor, fun stuff like that. Combining folds encoded as maps-and-foldl's is a bit tedious, but the Fold wrapper lets you do it in a cleaner first-class way using the abstractions everyone knows and loves.
For example, if i had
fold1 = map g . foldl' f z
and
fold2 = map g' . foldl' f' z'
and I wanted to do
fold3 = map (\(x,y) -> foo (g x) (g' y))
. foldl' (\(x,x') (y,y) -> (f x y, f' x' y')) (z', z')
(that is, do both folds on the list in one pas, and recombine the results at the end with foo). It's a big hassle, right?
But i can also just do
fold1 = Fold f z g
fold2 = Fold f' z' g'
fold3 = foo <$> fold1 <*> fold2
(Note that, even better, using using Fold actually keeps foldl' strict, because in the example above, the lazy tuples add a layer of indirection and make the fold' lazy again incidentally)
Presumably they do exactly the same thing, concatMap f xs and concat $ map f xs. Why would I choose one over another?
I imagine it may be an optimization. If so, is this still the case with GHC 7.8?
It is the case that concatMap f xs = concat (map f xs) as you suspect. Thus, for correctness purposes you should consider them interchangeable. We can examine their definitions to learn a little more, though.
concatMap :: (a -> [b]) -> [a] -> [b]
concatMap f = foldr ((++) . f) []
concat :: [[a]] -> [a]
concat = foldr (++) []
In particular, this means that concat . map f expands to foldr (++) [] . map f. Now using a thing known as the "universal property of fold" we can see that foldr g z . map f = foldr (g . f) z for any (g, z, f) such as the choice ((++), f, []) we use above. This demonstrates that concatMap f = concat . map f like we want.[0]
So why are they defined differently? Because foldr ((++) . f) [] is always going to be faster than foldr (++) [] . map f since, in a really pathological case, the latter suggests two separate recursions. Due to laziness, it's unlikely that two recursions would ever be performed, though, so what gives?
The real reason is that there are more complex fusion laws available to the compiler such as those which combine two sequential foldrs or which define interactions between foldr and unfoldr. These are kind of finicky to use as they depend upon being able to look at the surface syntax of a fragment of code and detect possible simplifications. A lot of work goes into getting consistently firing fusion laws.
But one thing we can do is encourage people to use higher order combinators with optimization laws pre-applied. Since foldr (++) [] . map f is never going to be faster than foldr ((++) . f) [] we can take a shortcut and pre-apply the universal law simplification. This will improve the likelihood of fusion laws firing elsewhere to best optimize a list production pipeline.
[0] Why does this law work? Roughly, the universal law of foldr states that if you have any function q such that q [] = z and q (a:as) = f a (q as) then that q must be and is foldr f z. Since q = foldr g z . map f can be shown to have q [] = z and q (a:as) = g (f a) (q as) then it must be a fold like foldr (g . f) z like we want.