I have a function with type below:
union :: a -> a -> a
And a has additivity property. So we can regard union as a version of (+)
Say, we have [a], and want to perform a parallel "folding", for non-parallel foldling we can do only:
foldl1' union [a]
But how to perform it in parallel?
I can demonstrate problem on Num values and (+) function.
For example, we have a list [1,2,3,4,5,6] and (+)
In parallel we should split
[1,2,3] (+) [4,5,6]
[1,2] (+) [3] (+) [4,5] (+) [6]
([1] (+) [2]) (+) ([3] (+) [4]) (+) ([5] (+) [6])
then each (+) operation we want to perform in parallel, and combine to answer
[3] (+) [7] (+) [11] = 21
Note, that we split list, or perform operations in any order, because of a additivity.
Is there any ways to do that using any standard library?
You need to generalize your union to any associative binary operator ⊕ such that (a ⊕ b) ⊕ c == a ⊕ (b ⊕ c). If at the same time you even have a unit element that is neutral with respect to ⊕, you have a monoid.
The important aspect of associativity is that you can arbitrarily group chunks of consecutive elements in a list, and ⊕ them in any order, since a ⊕ (b ⊕ (c ⊕ d)) == (a ⊕ b) ⊕ (c ⊕ d) - each bracket can be computed in parallel; then you'd need to "reduce" the "sums" of all brackets, and you've got your map-reduce sorted.
In order for this parallellisation to make sense, you need the chunking operation to be faster than ⊕ - otherwise, doing ⊕ sequentially is better than chunking. One such case is when you have a random access "list" - say, an array. Data.Array.Repa has plenty of parallellized folding functions.
If you are thinking of practicising to implement one yourself, you need to pick a good complex function ⊕ such that the benefit will show.
For example:
import Control.Parallel
import Data.List
pfold :: (Num a, Enum a) => (a -> a -> a) -> [a] -> a
pfold _ [x] = x
pfold mappend xs = (ys `par` zs) `pseq` (ys `mappend` zs) where
len = length xs
(ys', zs') = splitAt (len `div` 2) xs
ys = pfold mappend ys'
zs = pfold mappend zs'
main = print $ pfold (+) [ foldl' (*) 1 [1..x] | x <- [1..5000] ]
-- need a more complicated computation than (+) of numbers
-- so we produce a list of products of many numbers
Here I deliberately use a associative operation, which is called mappend only locally, to show it can work for a weaker notion than a monoid - only associativity matters for parallelism; since parallelism makes sense only for non-empty lists anyway, no need for mempty.
ghc -O2 -threaded a.hs
a +RTS -N1 -s
Gives 8.78 seconds total run time, whereas
a +RTS -N2 -s
Gives 5.89 seconds total run time on my dual core laptop. Obviously, no point trying more than -N2 on this machine.
What you've described is essentially a monoid. In GHCI:
Prelude> :m + Data.Monoid
Prelude Data.Monoid> :info Monoid
class Monoid a where
mempty :: a
mappend :: a -> a -> a
mconcat :: [a] -> a
As you can see a monoid has three associated functions:
The mempty function is sort of like the identity function of the monoid. For example a Num can behave as a monoid apropos two operations: sum and product. For a sum mempty is defined as 0. For a product mempty is defined as 1.
mempty `mappend` a = a
a `mappend` mempty = a
The mappend function is similar to your union function. For exampe for a sum of Nums mappend is defined as (+) and for a product of Nums mappend is defined as (*).
The mconcat function is similar to a fold. However because of the properties of a monoid it doesn't matter whether we fold from the left, fold from the right or fold from an arbitrary position. This is because mappend is supposed to be associative:
(a `mappend` b) `mappend` c = a `mappend` (b `mappend` c)
Note however that Haskell doesn't enforce the monoid laws. Hence if you make a type an instance of the Monoid typeclass then you're responsible to ensure that it satisfies the monoid laws.
In your case it's difficult to understand how union behaves from its type signature: a -> a -> a. Surely you can't make a type variable an instance of a typeclass. That's not allowed. You need to be more specific. What does union actually do?
To give you an example of how to make a type an instance of the monoid typeclass:
newtype Sum a = Sum { getSum :: a }
instance Num a => Monoid (Sum a) where
mempty = 0
mappend = (+)
That's it. We don't need to define the mconcat function because that has a default implementation that depends upon mempty and mappend. Hence when we define mempty and mappend we get mconcat for free.
Now you can use Sum as follows:
getSum . mconcat $ map Sum [1..6]
This is what's happening:
You're mapping the Sum constructor over [1..6] to produce [Sum 1, Sum 2, Sum 3, Sum 4, Sum 5, Sum 6].
You give the resulting list to mconcat which folds it to Sum 21.
You use getSum to extract the Num from Sum 21.
Note however that the default implementation of mconcat is foldr mappend mempty (i.e. it's a right fold). For most cases the default implementation is sufficient. However in your case you might want to override the default implementation:
foldParallel :: Monoid a => [a] -> a
foldParallel [] = mempty
foldParallel [a] = a
foldParallel xs = foldParallel left `mappend` foldParallel right
where size = length xs
index = (size + size `mod` 2) `div` 2
(left, right) = splitAt index xs
Now we can create a new instance of Monoid as follows:
data Something a = Something { getSomething :: a }
instance Monoid (Something a) where
mempty = unionEmpty
mappend = union
mconcat = foldParallel
We use it as follows:
getSomething . mconcat $ map Something [1..6]
Note that I defined mempty as unionEmpty. I don't know what type of data the union function acts on. Hence I don't know what mempty should be defined as. Thus I'm simply calling it unionEmpty. Define it as you see fit.
I know it's a long time after the OP, but I've just happened upon this and thought my experiences might be of help.
If we think about the problem, we can see that:
A fold is essentially a function that takes a list of items, and converts them to a single item which may be the same type as the items in the list, but doesn't have to be: so its type is ([a] -> b).
A parallel fold splits its input list into chunks, folds each chunk separately (in parallel), and then combines the results to derive the final result. For that we need:
A chunk size. This could be calculated with reference to the size of the input list, but that has a significant drawback: in order to determine the size of the list we have to process it, which loses the benefit of laziness. So it is better to make all chunks the same size; this could be hard-coded, but in a generic function it would be better to expose it as a parameter so that it can be varied and tuned to suit the needs of the calling application.
A function that knows how to combine results. This has the type (b -> b -> b).
A suitable generic parallel fold function is thus:
import Control.Parallel
foldParallel :: Int -> ([a] -> b) -> (b -> b -> b) -> [a] -> b
foldParallel _ fold _ [] = fold []
foldParallel chunkSize fold combine xs = par lf $ combine lf rf
where
(left, right) = splitAt chunkSize xs
lf = fold left
rf = foldParallel chunkSize fold combine right
The parallel processing is done explicitly, using the par function which kicks off the evaluation of its first operand, in parallel, and returns the second operand.
It took a while - for an ancient, imperative-programming dinosaur like me - to get my head around the fact that the definitions in the where block don't actually evaluate anything, but just set up things that can be evaluated; hence the fold named as lf can be referenced in both operands of par but is only evaluated once.
The difference that par makes is that if the function just returns combine lf rf, when that is evaluated lf needs to be evaluated, then rf, then combine lf rf. But par lf $ combine lf rf means that lf is already wholly or partly evaluated (in parallel) by the time its value is needed. And because rf is itself a parallel fold, the same is true of the folding of each subsequent chunk.
Related
The Foldr Foldl Foldl' wiki page describes the differences between foldr and foldl. Both process lists from left-to-right, but foldr accumulates the result from right-to-left whereas foldl does so from left-to-right.
The page goes on to discourage the use of foldl in favor of the eager version foldl', which is more efficient.
Is there a corresponding eager version of foldr, presumably called foldr'? If so, is there a reason that it isn't mentioned on the wiki page?
There's no need for foldr', sonce one can always use foldr f with a strict f to achieve the same goal.
We could define it...
foldr' f a xs = foldr f' a xs
where f' x y = seq x (seq y (f x y))
... but it's simpler to pass a strict f at the call point instead, if needed.
A quick Hoogle search shows that there is, indeed, a foldr' defined in Data.Foldable.
I assume the reason it isn't mentioned on the linked page is that it isn't relevant to the problem of stack overflows discussed there. When calculating a sum using foldr (+) 0 on a large list, the stack overflow that may occur isn't a result of lazy application of the operator (+). Rather, it occurs because a fundamental feature of a right fold (whether strict or lazy) is that the most deeply nested application of the operator occurs at the end of the list. For an operator like (+) that requires evaluation of both operands to yield any sort of result, this means that either foldr or foldr' must build up an O(n) stack of continuations before getting to the end of the list where the (+) operator can start to do "real work".
Both foldr and foldl' are extremely useful for lists. Neither foldl nor foldr' is often useful for lists. But these days, those functions are actually methods of the Foldable class, and they are useful for some instances. To take an extreme example, consider snoc lists:
data SL a = SL a :> a | Nil
infixl 5 :>
instance Foldable SL where
foldMap _ Nil = mempty
foldMap f (xs :> x) = foldMap f xs <> f x
foldl _ b Nil = b
foldl f b (xs :> x) = f (foldl f b xs) x
foldr' _ n Nil = n
foldr' c n (xs :> x) =
let n' = c x n
in n' `seq` foldr' c n' xs
For snoc lists, foldl and foldr' are really useful, while foldr and foldl' are pretty much useless!
Many Foldable containers can make good use of all four of these methods. For example, Data.Set.Set, Data.Map.Map k and Data.Sequence.Seq can each be folded perfectly well in either direction.
import Data.Sequence (Seq)
import qualified Data.Sequence as S
toForwardList :: Seq a -> [a]
toForwardList = foldr (:) []
toReverseList :: Seq a -> [a]
toReverseList = foldl (flip (:)) []
The forward and reverse conversions are equally efficient and equally lazy.
I am using the Control.Foldl library to traverse an arbitrarily long list and counting all occurrences of arbitrarily many unique entities. Ie, the list may be of form
[Just "a", Just "b", Just "aab", Nothing, Just "aab"]
and I my result should something like:
[(Just "a",1),(Just "b",1) (Just "aab", 2), (Nothing, 1)]
Now the issue is I do not have the name of these entities a priori, and I would like to dynamically update the results as I fold.
My problem is that I do not know how to describe this computation in terms of the Fold data type from Control.foldl. Specifically, at each step of the fold I need to traverse the result list and ask if I have seen the current item, but I see no way of describing this using foldl.
Please note for future use purposes it's really important that I use the Control.Foldl library here, not fold over some other foldable data type like a map. In some sense my question is more along the lines of how to use the Foldl library, since the documentation is not too clear to me.
Edit: The example I showed is just a toy example, in reality I need to traverse a arb large list many times computing statistics, hence I'm using the foldl library, which allow me to combine the computations using applicatives ie toResults <$> stat1 <*> stat2 <*> ... <*> statm $ largeList and foldl allow me to traverse the list just once, computing all m statistics. Please find a solution using the foldl library.
You can encode a normal foldl' pretty straightforwardly as a Fold:
foldlToFold :: (b -> a -> b) -> b -> Fold a b
foldlToFold f z = Fold f z id
I'm actually a bit puzzled that this combinator isn't in the library...
Anyways, if you have
foldl' f z
you can replace it with
fold (Fold f z id)
so here, you would normally be using
foldl' (\mp x -> M.insertWith (+) x 1 mp) M.empty
with Fold, you'd be making
countingFold :: Ord a => Fold a (Map a Int)
countingFold = Fold (\mp x -> M.insertWith (+) 1 mp) M.empty id
and you can use it as
countUp :: Ord a => [a] -> Map a Int
countUp = fold countingFold
-- or
countUp = fold (Fold (\mp x -> M.insertWith (+) 1 mp) M.empty id)
If you want to go back to a list at the end, you can do
M.toList . countUp
In general, if you can formulate your fold as a foldl', you can do the transformation above to be able to encode it as a Fold. Fold is a bit more expressive because for foldl', the b type is both the accumulator and the result type; for a Fold, you can have a separate accumulator and result type.
Roughly speaking, you can translate any Fold into a foldl-and-map:
Fold f z g = map g . foldl' f z
And you can go backwards too:
foldlMapToFold :: (b -> a -> b) -> b -> (b -> c) -> Fold a c
foldlMapToFold = Fold
So if you had
map g . foldl' f z
you can write
fold (Fold f z g)
If you want to use a Fold, think, "how can i describe my operation as a foldl' and a map?", and then go from there.
The advantage of using the Fold type over just normal maps and folds is (apart from performance tweaks) the ability to combine and manipulate multiple Folds as objects using their Applicative instance, and other nice instances too, like Functor, Profunctor, fun stuff like that. Combining folds encoded as maps-and-foldl's is a bit tedious, but the Fold wrapper lets you do it in a cleaner first-class way using the abstractions everyone knows and loves.
For example, if i had
fold1 = map g . foldl' f z
and
fold2 = map g' . foldl' f' z'
and I wanted to do
fold3 = map (\(x,y) -> foo (g x) (g' y))
. foldl' (\(x,x') (y,y) -> (f x y, f' x' y')) (z', z')
(that is, do both folds on the list in one pas, and recombine the results at the end with foo). It's a big hassle, right?
But i can also just do
fold1 = Fold f z g
fold2 = Fold f' z' g'
fold3 = foo <$> fold1 <*> fold2
(Note that, even better, using using Fold actually keeps foldl' strict, because in the example above, the lazy tuples add a layer of indirection and make the fold' lazy again incidentally)
In Haskell, the default ordering for [a], given an ordering on a, seems to be a lexicographic ordering (side question: where can I find out if this is really the case)? What I want is a graded lexicographic ordering (also called "length plus lexicographic" ordering).
How would I specify that I want comparisons to be done in a graded lexicographical manner? I want it for only one type, not for all [a]. I tried this:
instance Ord [Int] where
compare xs ys = case compare (length xs) (length ys) of
LT -> LT
GT -> GT
EQ -> lexicographic_compare xs ys
but got this error message:
> [1 of 1] Compiling Main ( test.hs, interpreted )
test.hs:1:10:
Illegal instance declaration for `Ord [Int]'
(All instance types must be of the form (T a1 ... an)
where a1 ... an are *distinct type variables*,
and each type variable appears at most once in the instance head.
Use -XFlexibleInstances if you want to disable this.)
In the instance declaration for `Ord [Int]'
Failed, modules loaded: none.
Thanks for any and all help!
This is a typical application for a newtype wrapper:
newtype GradedLexOrd a = GradedLexOrd { runGradedLexOrd :: [a] }
instance (Ord a) => Ord (GradedLexOrd a) where
compare (GradedLexOrd xs) (GradedLexOrd ys) = gradedLexOrd xs ys
gradedLexOrd :: Ord a => [a] -> [a] -> Ordering
gradedLexOrd = comparing length <> compare -- Nice Monoid-based implementation,
--due to Aaron Roth (see answer below)
Alternatively, you could openly use lists, but instead of the Ord constrained functions like sort use the more general alternatives which accept a custom comparison function, e.g. sortBy gradedLexOrd.
There are two questions here:
How does Ord [a] looks like?
Of course you can experiment within GHCi, but maybe you want something more reliable. This is surprisingly difficult, especially as the definition of Lists is (due to their special syntax) built into the compiler. Let’s ask GHCi:
Prelude> :info []
data [] a = [] | a : [a] -- Defined in `GHC.Types'
instance Eq a => Eq [a] -- Defined in `GHC.Classes'
instance Monad [] -- Defined in `GHC.Base'
instance Functor [] -- Defined in `GHC.Base'
instance Ord a => Ord [a] -- Defined in `GHC.Classes'
instance Read a => Read [a] -- Defined in `GHC.Read'
instance Show a => Show [a] -- Defined in `GHC.Show'
It says that the instance is defined in GHC.Classes, which we find in GHC’s git repo, and there it says:
instance (Ord a) => Ord [a] where
{-# SPECIALISE instance Ord [Char] #-}
compare [] [] = EQ
compare [] (_:_) = LT
compare (_:_) [] = GT
compare (x:xs) (y:ys) = case compare x y of
EQ -> compare xs ys
other -> other
So yes, it is indeed the lexicographic ordering.
How to overwrite the ordering?
Don’t. There is an instance for [a] and there can be only one. With FlexibleInstances and OverlappingInstances, you could make it use an alternative instance for, say, [Int], but it is bad style. As leftaroundabout writes, use a NewtypeWrapper for it, or use parametrized functions like sortBy.
Creating a whole new Ord instance for lists of Ints seems a bit heavyweight to my taste (not to mention that you may be sowing confusion: someone who comes along to your code later will probably expect the default, non-graded lexicographic comparison behavior).
If you're merely hoping not to have to copy your custom comparison code every time you use sortBy or the like, there's actually a fairly lightweight way of defining chained comparison functions like yours on the spot. Ordering, as it happens, is an instance of Monoid, which means you can compare two things according to a succession of criteria, then combine the resulting Orderings of those comparisons using the Monoid function, mappend (recently abbreviated to <>). This is all explained in some detail in the Learn You a Haskell chapter on Monoids, etc., which is where I picked up the trick. So:
import Data.Monoid ((<>))
import Data.Ord (comparing)
gradedLexicographicCompare :: (Ord a) => [a] -> [a] -> Ordering
gradedLexicographicCompare xs ys = comparing length xs ys <> comparing id xs ys
(Of course, comparing id is just compare, but for the sake of uniformity...) Then it becomes relatively unburdensome to write things like
f = ... sortBy s ...
where
...
s xs ys = comparing length xs ys <> compare xs ys
...
And this also has the virtue that your successor will see immediately that you're using a custom comparison function.
Update: leftaroundabout points out below that we can achieve even greater elegance -- this is Haskell after all, and in Haskell we can always achieve greater elegance -- by making use of the monoid instance, instance Monoid b => Monoid (a -> b). That is, a function whose result is a monoid can itself be considered a monoid. The instance is given by
instance Monoid b => Monoid (a -> b) where
mempty _ = mempty
mappend f g x = f x `mappend` g x (1)
Now let's indulge in a little equational reasoning and see what comparing length <> compare expands to according to this instance. Applying (1) once, we have
comparing length <> compare
= mappend (comparing length) compare
= \xs -> mappend ((comparing length) xs) (compare xs) (2)
But ((comparing length) xs) :: [a] -> Ordering and (compare xs) :: (Ord a) => a -> Ordering are themselves functions whose results are monoids, namely Orderings, so we can apply (1) a second time to obtain
mappend ((comparing length) xs) (compare xs)
= \ys -> mappend (((comparing length) xs) ys) ((compare xs) ys) (3)
But now (((comparing length) xs) ys) and ((compare xs) ys) are fully applied functions. Specifically, they are Orderings, and from the original answer we know how to combine two Orderings using mappend from the Ordering instance of Monoid. (Note that we are not using mappend from (1).) Writing everything down in one big chain, we have
comparing length <> compare
= mappend (comparing length) compare [definition of <>]
= \xs -> mappend ((comparing length) xs) (compare xs) [by (1)]
= \xs -> (\ys -> mappend (((comparing length) xs) ys) ((compare xs) ys)) [substituting (3) in (2)]
= \xs -> \ys -> mappend (comparing length xs ys) (compare xs ys) [function application is left associative]
= \xs -> \ys -> comparing length xs ys <> compare xs ys [definition of <>]
And the last line of this expansion is just our original gradedLexicographicCompare! After a long, long digression, then, the punchline is that we can write the elegantly points-free
gradedLexicographicCompare = comparing length <> compare
Pretty.
So I keep hearing a lot about point free programming and I decided to do a little experiment to test my grasp of it. This involved taking a pointed function to calculate the factorial of a number and converting it to a point-free form. I managed to do it, but the point free result is a lot less readable than the pointed result.
-- pointed
fact 0 = 1
fact n = n * (fact (n-1))
-- point free
fact' = foldr1 (*) . takeWhile ((<) 0) . iterate (flip (-) 1)
Am I missing something essential to point free notation or is this as readable as certain transformations get? To me it seems that a big part of the fact function is the pattern match on zero, and indeed, pattern matching is one of the biggest reasons I love Haskell. However point free notation seems to completely disallow that, along with some other things that are extremely useful like list comprehensions.
The canonical factorial in pointfree form is:
fact = product . enumFromTo 1
(which is equivalent to fact n = product [1..n])
I find this to be pretty readable. However, I would concur that the original version:
fact 0 = 1
fact n = n * (fact (n-1))
Matches the definition very well and is also readable.
The point (ha!) of pointfree form is to make it easy to reason about functions as the composition of other functions. However, the factorial function isn't really an excellent candidate for this kind of reasoning.
The decision is yours, obviously.
For each algebraic union data type there should exist its type case discriminator function which encapsulates the pattern matching for that type. We already have
either :: (a -> c) -> (b -> c) -> Either a b -> c
maybe :: b -> (a -> b) -> Maybe a -> b
Similarly there must be such function for numbers,
num :: (Num a) => b -> (a -> b) -> a -> b
num z nz 0 = z
num z nz x = nz x
so we can write
import Control.Applicative
import Data.Function
fact :: (Num a) => a -> a
fact x = num 1 (\x-> (*) (fact (pred x)) x) x
= num 1 ((*) =<< (fact.pred)) x
i.e.
fact = (num 1 . ((*) =<<) . (. pred)) fact
= fix (num 1 . ((*) =<<) . (. pred))
I can define a natural transformation in Haskell as:
h :: [a] -> Maybe a
h [] = Nothing
h (x:_) = Just x
and with a function k:
k :: Char -> Int
k = ord
the naturality condition is met due to the fact that:
h . fmap k == fmap k . h
Can the naturality condition of the List monad's join function be demonstrated in a similar way? I'm having some trouble understanding how join, say concat in particular, is a natural transformation.
Okay, let's look at concat.
First, here's the implementation:
concat :: [[a]] -> [a]
concat = foldr (++) []
This parallels the structure of your h where Maybe is replaced by [] and, more significantly, [] is replaced by--to abuse syntax for a moment--[[]].
[[]] is a functor as well, of course, but it's not a Functor instance in the way that the naturality condition uses it. Translating your example directly won't work:
concat . fmap k =/= fmap k . concat
...because both fmaps are working on only the outermost [].
And although [[]] is hypothetically a valid instance of Functor you can't make it one directly, for practical reasons that are probably obvious.
However, you can reconstruct the correct lifting as so:
concat . (fmap . fmap) k == fmap k . concat
...where fmap . fmap is equivalent to the implementation of fmap for a hypothetical Functor instance for [[]].
As a related addendum, return is awkward for the opposite reason: a -> f a is a natural transformation from an elided identity functor. Using : [] the identity would be written as so:
(:[]) . ($) k == fmap k . (:[])
...where the completely superfluous ($) is standing in for what would be fmap over the elided identity functor.