Does Haskell standard library have a function that given a list and a predicate, returns the number of elements satisfying that predicate? Something like with type (a -> Bool) -> [a] -> Int. My hoogle search didn't return anything interesting. Currently I am using length . filter pred, which I don't find to be a particularly elegant solution. My use case seems to be common enough to have a better library solution that that. Is that the case or is my premonition wrong?
The length . filter p implementation isn't nearly as bad as you suggest. In particular, it has only constant overhead in memory and speed, so yeah.
For things that use stream fusion, like the vector package, length . filter p will actually be optimized so as to avoid creating an intermediate vector. Lists, however, use what's called foldr/build fusion at the moment, which is not quite smart enough to optimize length . filter p without creating linearly large thunks that risk stack overflows.
For details on stream fusion, see this paper. As I understand it, the reason that stream fusion is not currently used in the main Haskell libraries is that (as described in the paper) about 5% of programs perform dramatically worse when implemented on top of stream-based libraries, while foldr/build optimizations can never (AFAIK) make performance actively worse.
No, there is no predefined function that does this, but I would say that length . filter pred is, in fact, an elegant implementation; it's as close as you can get to expressing what you mean without just invoking the concept directly, which you can't do if you're defining it.
The only alternatives would be a recursive function or a fold, which IMO would be less elegant, but if you really want to:
foo :: (a -> Bool) -> [a] -> Int
foo p = foldl' (\n x -> if p x then n+1 else n) 0
This is basically just inlining length into the definition. As for naming, I would suggest count (or perhaps countBy, since count is a reasonable variable name).
Haskell is a high-level language. Rather than provide one function for every possible combination of circumstances you might ever encounter, it provides you with a smallish set of functions that cover all of the basics, and you then glue these together as required to solve whatever problem is currently at hand.
In terms of simplicity and conciseness, this is as elegant as it gets. So yes, length . filter pred is absolutely the standard solution. As another example, consider elem, which (as you may know) tells you whether a given item is present in a list. The standard reference implementation for this is actually
elem :: Eq x => x -> [x] -> Bool
elem x = foldr (||) False . map (x ==)
In order words, compare every element in the list to the target element, creating a new list of Bools. Then fold the logical-OR function over this new list.
If this seems inefficient, try not to worry about it. In particular,
The compiler can often optimise away temporary data structures created by code like this. (Remember, this is the standard way to write code in Haskell, so the compiler is tuned to deal with it.)
Even if it can't be optimised away, laziness often makes such code fairly efficient anyway.
(In this specific example, the OR function will terminate the loop as soon as a match is seen - just like what would happen if you hand-coded it yourself.)
As a general rule, write code by gluing together pre-existing functions. Change this only if performance isn't good enough.
This is my amateurish solution to a similar problem. Count the number of negative integers in a list l
nOfNeg l = length(filter (<0) l)
main = print(nOfNeg [0,-1,-2,1,2,3,4] ) --2
No, there isn't!
As of 2020, there is indeed no such idiom in the Haskell standard library yet! One could (and should) however insert an idiom howMany (resembling good old any)
howMany p xs = sum [ 1 | x <- xs, p x ]
-- howMany=(length.).filter
main = print $ howMany (/=0) [0..9]
Try howMany=(length.).filter
I'd do manually
howmany :: (a -> Bool) -> [a] -> Int
howmany _ [ ] = 0
howmany pred (x:xs) = if pred x then 1 + howmany pred xs
else howmany pred xs
Related
I am new to Haskell and I am trying the below code to remove duplicates from a list. However it does not seem to work.
compress [] = []
compress (x:xs) = x : (compress $ dropWhile (== x) xs)
I have tried some search, all the suggestions use foldr/ map.head. Is there any implementation with basic constructs?
I think that the issue that you are referring to in your code is that your current implementation will only get rid of adjacent duplicates. As it was posted in a comment, the builtin function nub will eliminate every duplicate, even if it's not adjacent, and keep only the first occurrence of any element. But since you asked how to implement such a function with basic constructs, how about this?
myNub :: (Eq a) => [a] -> [a]
myNub (x:xs) = x : myNub (filter (/= x) xs)
myNub [] = []
The only new function that I introduced to you there is filter, which filters a list based on a predicate (in this case, to get rid of every element present in the rest of that list matching the current element).
I hope this helps.
First of all, never simply state "does not work" in a question. This leaves to the reader to check whether it's a compile time error, run time error, or a wrong result.
In this case, I am guessing it's a wrong result, like this:
> compress [1,1,2,2,3,3,1]
[1,2,3,1]
The problem with your code is that it removes successive duplicates, only. The first pair of 1s gets compressed, but the last lone 1 is not removed because of that.
If you can, sort the list in advance. That will make equal elements close, and then compress does the right job. The output will be in a different order, of course. There are ways to keep the order too if needed (start with zip [0..] xs, then sort, then ...).
If you can not sort becuase there is really no practical way to define a comparison, but only an equality, then use nub. Be careful that this is much less efficient than sorting & compressing. This loss of performance is intrinsic: without a comparator, you can only use an inefficient quadratic algorithm.
foldr and map are very basic FP constructs. However, they are very general and I found them a bit mind-bending to understand for a long time. Tony Morris' talk Explain List Folds to Yourself helped me a lot.
In your case an existing list function like filter :: (a -> Bool) -> [a] -> [a] with a predicate that exludes your duplicate could be used in lieu of dropWhile.
I have a function that will take and int and return its square root. However now i want to modify it so that it takes an array of integers and gives back an array with the square roots of the elements of the first array. I know Haskell does not use loops so how can this modification be done? Thanks.
intSquareRoot :: Int -> Int
intSquareRoot n = try n where
try i | i*i > n = try (i - 1)
| i*i <= n = i
Don't.
The idea of “looping through some collection”, putting each result in the corresponding slot of its input, is a somewhat trivial, extremely common pattern. Patterns are for OO programmers. In Haskell, when there's a pattern, we want to abstract over it, i.e. give it a simple name that we can always re-use without extra boilerplate.
This particular “pattern” is the functor operation1. For lists it's called
map :: (a->b) -> [a]->[b]
more generally (e.g. it'll also work with real arrays; lists aren't actually arrays),
class Functor f where
fmap :: (a->b) -> f a->f b
So instead of defining an extra function
intListSquareRoot :: [Int] -> [Int]
intListSquareRoot = ...
you simply use map intSquareRoot right where you wanted to use that function.
Of course, you could also define that “lifted” version of intSquareRoot,
intListSquareRoot = map intSquareRoot
but that gains you practically nothing over simply inlining the map call right where you need it.
If you insist
That said... it's of course valid to wonder how map itself works. Well, you can manually “loop” through a list by recursion:
map' :: (a->b) -> [a]->[b]
map' _ [] = []
map' f (x:xs) = f x : map' f xs
Now, you could inline your specific function here
intListSquareRoot' :: [Int] -> [Int]
intListSquareRoot' [] = []
intListSquareRoot' (x:xs) = intSquareRoot x : intListSquareRoot' xs
This is not only much more clunky and awkward than quickly inserting the map magic word, it will also often be slower: compilers such as GHC can make better optimisations when they work on higher-level concepts2 such as folds, than when they have to work again and again with manually defined recursion.
1Not to be confused what many C++ programmers call a “functor”. Haskell uses the word in the correct mathematical sense, which comes from category theory.
2This is why languages such as Matlab and APL actually achieve decent performance for special applications, although they are dynamically-typed, interpreted languages: they have this special case of “vector looping” hard-coded into their very syntax. (Unfortunately, this is pretty much the only thing they can do well...)
You can use map:
arraySquareRoot = map intSquareRoot
I am currently facing the problem of having to make my calculations based on the length of a given list. Having to iterate over all the elements of the list to know its size is a big performance penalty as I'm using rather big lists.
What are the suggested approaches to the problem?
I guess I could always carry a size value together with the list so I know beforehand its size without having to compute it at the call site but that seems a brittle approach. I could also define a own type of list where each node has as property its the lists' size but then I'd lose the leverage provided by my programming language's libraries for standard lists.
How do you guys handle this on your daily routine?
I am currently using F#. I am aware I can use .NET's mutable (array) lists, which would solve the problem. I am way more interested, though, in the purely immutable functional approach.
The built-in F# list type doesn't have any caching of the length and there is no way to add that in some clever way, so you'll need to define your own type. I think that writing a wrapper for the existing F# list type is probably the best option.
This way, you can avoid explicit conversions - when you wrap the list, it will not actually copy it (as in svick's implementation), but the wrapper can easily cache the Length property:
open System.Collections
type LengthList<'T>(list:list<'T>) =
let length = lazy list.Length
member x.Length = length.Value
member x.List = list
interface IEnumerable with
member x.GetEnumerator() = (list :> IEnumerable).GetEnumerator()
interface seq<'T> with //'
member x.GetEnumerator() = (list :> seq<_>).GetEnumerator()
[<CompilationRepresentation(CompilationRepresentationFlags.ModuleSuffix)>]
module LengthList =
let ofList l = LengthList<_>(l)
let ofSeq s = LengthList<_>(List.ofSeq s)
let toList (l:LengthList<_>) = l.List
let length (l:LengthList<_>) = l.Length
The best way to work with the wrapper is to use LengthList.ofList for creating LengthList from a standard F# list and to use LengthList.toList (or just the List) property before using any functions from the standard List module.
However, it depends on the complexity of your code - if you only need length in a couple of places, then it may be easier to keep it separately and use a tuple list<'T> * int.
How do you guys handle this on your daily routine?
We don't, because this isn't a problem in daily routine. It sounds like a problem perhaps in limited domains.
If you created the lists recently, then you've probably already done O(N) work, so walking the list to get its length is probably not a big deal.
If you're making a few very large lists that are then not 'changing' much (obviously never changing, but I mean changing set of references to heads of lists that are used in your domain/algorithm), then it may make sense to just have a dictionary off to the side of reference-to-list-head*length tuples, and consult the dictionary when asking for lengths (doing the real work to walk them when needed, but caching results for future asks about the same list).
Finally, if you really are dealing with some algorithm that needs to be constantly updating the lists in play and constantly consulting the lengths, then create your own list-like data type (yes, you'll also need to write map/filter and any others).
(Very generally, I think it is typically best to use the built-in data structures 99.99% of the time. In the 0.01% of the time where you are developing an algorithm or bit of code that needs to be very highly optimized, then almost always you need to abandon built-in data structures (which are good enough for most cases) and use a custom data structure designed to solve the exact problem you are working on. Look to wikipedia or Okasaki's "'Purely Functional Data Structures" for ideas and inspriation in that case. But rarely go to that case.)
I don't see why carying the length around is a brittle approach. Try something like this (Haskell):
data NList a = NList Int [a]
nNil :: NList [a]
nNil = NList 0 []
nCons :: a -> NList a -> NList a
nCons x (NList n xs) = NList (n+1) (x:xs)
nHead :: NList a -> a
nHead (NList _ (x:_)) = x
nTail :: NList a -> NList a
nTail (NList n (_:xs)) = NList (n-1) xs
convert :: [a] -> NList a
convert xs = NList (length xs) xs
and so on. If this is in a library or module, you can make it safe (I think) by not exporting the constructor NList.
It may also be possible to coerce GHC into memoizing length, but I'm not sure how or when.
In F#, most List functions have an equivalent Seq functions. That means, you can just implement your own immutable linked list that carries the length with each node. Something like this:
type MyList<'T>(item : Option<'T * MyList<'T>>) =
let length =
match item with
| None -> 0
| Some (_, tail) -> tail.Length + 1
member this.Length = length
member private this.sequence =
match item with
| None -> Seq.empty
| Some (x, tail) ->
seq {
yield x
yield! tail.sequence
}
interface seq<'T> with
member this.GetEnumerator() =
(this.sequence).GetEnumerator()
member this.GetEnumerator() =
(this.sequence :> System.Collections.IEnumerable).GetEnumerator()
module MyList =
let rec ofList list =
match list with
| [] -> MyList None
| head::tail -> MyList(Some (head, ofList tail))
I am not very familiar with the degree that Haskell/GHC can optimize code. Below I have a pretty "brute-force" (in the declarative sense) implementation of the n queens problem. I know it can be written more efficiently, but thats not my question. Its that this got me thinking about the GHC optimizations capabilities and limits.
I have expressed it in what I consider a pretty straightforward declarative sense. Filter permutations of [1..n] that fulfill the predicate For all indices i,j s.t j<i, abs(vi - vj) != j-i I would hope this is the kind of thing that can be optimized, but it also kind of feels like asking a lot of compiler.
validQueens x = and [abs (x!!i - x!!j) /= j-i | i<-[0..length x - 2], j<-[i+1..length x - 1]]
queens n = filter validQueens (permutations [1..n])
oneThru x = [1..x]
pointlessQueens = filter validQueens . permutations . oneThru
main = do
n <- getLine
print $ pointlessQueens $ (read :: String -> Int) n
This runs fairly slow and grows quickly. n=10 takes about a second and n=12 takes forever. Without optimization I can tell the growth is factorial (# of permutations) multiplied by quadratic (# of differences in the predicate to check). Is there any way this code can perform better thru intelligent compilation? I tried the basic ghc options such has -O2 and didn't notice a significant difference, but I don't know the finer points (just added the flagS)
My impression is that the function i call queens can not be optimized and must generate all permutations before filter. Does the point-free version have a better chance? On the one hand I feel like a smart function comprehension between a filter and a predicate might be able to knock off some obviously undesired elements before they are even fully generated, but on the other hand it kind of feels like a lot to ask.
Sorry if this seems rambling, i guess my question is
Is the pointfree version of above function more capable of being optimized?
What steps could I take at make/compile/link time to encourage optimization?
Can you briefly describe some possible (and contrast with the impossible!) means of optimization for the above code? At what point in the process do these occur?
Is there any particular part of ghc --make queensN -O2 -v output I should be paying attention to? Nothing stands out to me. Don't even see much difference in output due to optimization flags
I am not overly concerned with this code example, but I thought writing it got me thinking and it seems to me like a decent vehicle for discussing optimization.
PS - permutations is from Data.List and looks like this:
permutations :: [a] -> [[a]]
permutations xs0 = xs0 : perms xs0 []
where
perms [] _ = []
perms (t:ts) is = foldr interleave (perms ts (t:is)) (permutations is)
where interleave xs r = let (_,zs) = interleave' id xs r in zs
interleave' _ [] r = (ts, r)
interleave' f (y:ys) r = let (us,zs) = interleave' (f . (y:)) ys r
in (y:us, f (t:y:us) : zs)
At a more general level regarding "what kind of optimizations can GHC do", it may help to break the idea of an "optimization" apart a little bit. There are conceptual distinctions that can be drawn between aspects of a program that can be optimized. For instance, consider:
The intrinsic logical structure of the algorithm: You can safely assume in almost every case that this will never be optimized. Outside of experimental research, you're not likely to find a compiler that will replace a bubble sort with a merge sort, or even an insertion sort, and extremely unlikely to find one that would replace a bogosort with something sensible.
Non-essential logical structure of the algorithm: For instance, in the expression g (f x) (f x), how many times will f x be computed? What about an expression like g (f x 2) (f x 5)? These aren't intrinsic to the algorithm, and different variations can be interchanged without impacting anything other than performance. The difficulties in performing optimization here are essentially recognizing when a substitution can in fact be done without changing the meaning, and predicting which version will have the best results. A lot of manual optimizations fall into this category, along with a great deal of GHC's cleverness.
This is also the part that trips a lot of people up, because they see how clever GHC is and expect it to do even more. And because of the reasonable expectation that GHC should never make things worse, it's not uncommon to have potential optimizations that seem obvious (and are, to the programmer) that GHC can't apply because it's nontrivial to distinguish cases where the same transformation would significantly degrade performance. This is, for instance, why memoization and common subexpression elimination aren't always automatic.
This is also the part where GHC has a huge advantage, because laziness and purity make a lot of things much easier, and is I suspect what leads to people making tongue-in-cheek remarks like "Optimizing compilers are a myth (except perhaps in Haskell).", but also to unrealistic optimism about what even GHC can do.
Low-level details: Things like memory layout and other aspects of the final code. These tend to be somewhat arcane and highly dependent on implementation details of the runtime, the OS, and the processor. Optimizations of this sort are essentially why we have compilers, and usually not something you need to worry about unless you're writing code that is very computationally demanding (or are writing a compiler yourself).
As far as your specific example here goes: GHC isn't going to significantly alter the intrinsic time complexity of your algorithm. It might be able to remove some constant factors. What it can't do is apply constant-factor improvements that it can't be sure are correct, particularly ones that technically change the meaning of the program in ways that you don't care about. Case in point here is #sclv's answer, which explains how your use of print creates unnecessary overhead; there's nothing GHC could do about that, and in fact the current form would possibly inhibit other optimizations.
There's a conceptual problem here. Permutations is generating streaming permutations, and filter is streaming too. What's forcing everything prematurely is the "show" implicit in "print". Change your last line to:
mapM print $ pointlessQueens $ (read :: String -> Int) n
and you'll see that results are generated in a streaming fashion much more rapidly. That fixes, for large result sets, a potential space leak, and other than that just lets things be printed as computed rather than all at once at the end.
However, you shouldn't expect any order of magnitude improvements from ghc optimizations (there are a few, obvious ones that you do get, mostly having to do with strictness and folds, but its irritating to rely on them). What you'll get are constant factors, generally.
Edit: As luqui points out below, show is also streaming (or at least show of [Int] is), but the line buffering nonetheless makes it harder to see the genuine speed of computation...
It should be noted, although you do express that it is not part of your question, that the big problem with your code is that you do not do any pruning.
In the case of your question, it feels foolish to talk about possible/impossible optimization, compiler flags, and how to best formulate it etc. when an improvement of the algorithm is staring us so blatantly in the face.
One of the first things that will be tried is the permutations starting with the first queen in position 1 and the second queen in position 2 ([1,2...]). This is of course not a solution and we will have to move one of the queens. However, in your implementation, all permutations involving this combination of the two first queens will be tested! The search should stop there and instantly move to the permutations involving [1,3,...].
Here is a version that does this sort of pruning:
import Data.List
import Control.Monad
main = getLine >>= mapM print . queens . read
queens :: Int -> [[Int]]
queens n = queens' [] n
queens' xs n
| length xs == n = return xs
| otherwise = do
x <- [1..n] \\ xs
guard (validQueens (x:xs))
queens' (x:xs) n
validQueens x =
and [abs (x!!i - x!!j) /= j-i | i<-[0..length x - 2], j<-[i+1..length x - 1]]
I understand that your question was about compiler optimization but as the discussion has shown pruning is necessary.
The first paper that I know of about how to do this for the n queens problem in a lazy functional language is Turner's paper "Recursion Equations as a Programming Language" You can read it in Google Books here.
In terms of your comment about a pattern worth remembering, this problem introduces a very powerful pattern. A great paper on this idea is Philip Wadler's paper, "How to Replace Failure by a List of Successes", which can be read in Google Books here
Here is a pure, non-monadic, implementation based on Turner's Miranda implementation. In the case of n = 12 (queens 12 12) it returns the first solution in .01 secs and will compute all 14,200 solutions in under 6 seconds. Of course printing those takes much longer.
queens :: Int -> Int -> [[Int]]
queens n boardsize =
queensi n
where
-- given a safe arrangement of queens in the first n - 1 rows,
-- "queensi n" returns a list of all the safe arrangements of queens
-- in the first n rows
queensi :: Int -> [[Int]]
queensi 0 = [[]]
queensi n = [ x : y | y <- queensi (n-1) , x <- [1..boardsize], safe x y 1]
-- "safe x y n" tests whether a queen at column x would be safe from previous
-- queens in y where the first element of y is n rows away from x, the second
-- element is (n+1) rows away from x, etc.
safe :: Int -> [Int] -> Int -> Bool
safe _ [] _ = True
safe x (c:y) n = and [ x /= c , x /= c + n , x /= c - n , safe x y (n+1)]
-- we only need to check for queens in the same column, and the same diagonals;
-- queens in the same row are not possible by the fact that we only pick one
-- queen per row
I am trying to make a function that outputs char*m n times, as such as the expected output would be ["ccc","ccc"] for the input 2 3 c. Here is what i have so far:
rectangle :: Int -> Int -> Char -> [string]
rectangle n m c
| m > 0 = [concat ([[c]] ++ (rectangle n (m-1) c))]
| otherwise = []
I am able to carry out the first part, char*m, so it returns ["ccc"]. Thing is: I also would like to be able to repeat my string n times.
I have tried using replicate but it doesn't seem to work, yet it works if doing it in the console: replicate 2 (rectangle 2 3 c).
Try the replicate function this way:
replicate :: Int -> a -> [a]
rectangle n m c = replicate n (replicate m c)
Also, don't forget to mention if this is homework.
As an addendum to Refactor's answer, I think his approach is the correct one. He subdivides the problem until it can be solved trivially using built-in functions. If you want to roll your own solution for learning purposes, I suggest you keep this subdivision, and go from there, implementing your own replicate. Otherwise, you will end up with a single function which does too much.
So the remaining problem is that of implementing replicate. My first idea would be to look at the source code for replicate. I found it via hoogle, which led me to hackage, which has links to the source code. Excerpted from the source:
replicate :: Int -> a -> [a]
replicate n x = take n (repeat x)
which is nice and concise, again using the built-in functions. If you want to completely roll your own replicate, you can do:
myReplicate :: Int -> a -> [a]
myReplicate n x | n <= 0 = []
| otherwise = x : replicate (n-1) x
----------EDIT----------------
As a side note, I think your problem requires two rather orthogonal skills. The first is trying not to tackle the whole problem at once, but making some small progress instead. Then you can try to solve that smaller problem, before returning to the larger. In your case, it would likely involve recognizing that you definitely need a way of transforming the character into a series of characters of length n. Experience with functions such as map, filter, foldr and so on will help you here, since they each represent a very distinct transformation, which you might recognize.
The second skill required for your solution - if you want to roll your own - is recognizing when a function can be expressed recursively. As you can see, your problem - and indeed many common problems - can be solved without explicit recursion, but it is a nice skill to have, when the need arises. Recursive solutions do not always come easily mind, so I think the best way to gain familiarity with them are to read and practice.
For further study, I'm sure you have already been pointed to the excellent Learn You a Haskell and Real World Haskell, but just in case you haven't, here they are.