I'm a beginner to Haskell and used it to solve some 50 problems of Project Euler but now I'm stuck at problem 66. The problem is that the compiled code (ghc -O2 --make problem66.hs) takes all my machine's free memory after 10-20 seconds. My code looks like this:
-- Project Euler, problem 66
diophantine x y d = x^2 - d*y^2 == 1
minimalsolution d = take 1 [(x, y, d) | y <- [2..],
let x = round $ sqrt $ fromIntegral (d*y^2+1),
diophantine x y d]
issquare x = (round $ sqrt $ fromIntegral x)^2 == x
main = do
print (map minimalsolution (filter (not . issquare) [1..1000]))
I have a hunch that the problem lies in the infinite list inside the list comprehension for minimalsolution.
I actually thought that due to lazyness, Haskell would evaluate the list only until it finds one element (because of take 1) and on the way discard everything for which diophantine evaluates to False. Am I wrong there?
Interestingly, I did not see this behaviour in ghci. Is it because processing inside ghci is so much slower that I just would have to wait until I see the memory consumption explode - or is it something else?
No spoilers, please. All I want to know is where the extreme memory consumption comes from and how I can fix it.
I haven't profiled before, so stone throwers are welcome.
Haskell determines that [2..] is a constant and is reused for every element of the list, despite take 1 using only one element of that list; so it memoizes the list for computing future elements of the same list. You get stuck computing value for d=61.
Edit:
What's interesting, this one terminates for [1..1000]:
minimalsolution d = take 1 [(x, y, d) | y <- [2..] :: [Int],
let x = round $ sqrt $ fromIntegral (d*y^2+1),
diophantine x y d]
Just added :: [Int]. Memory use looks stable at 1MB. Using Int64 reproduces the problem.
minimalsolution d = take 1 [(x, y, d) | y <- [2..] :: [Int64],
let x = round $ sqrt $ fromIntegral (d*y^2+1),
diophantine x y d]
Edit:
Well, as has been suggested, the difference is caused by overflow. The solution to d=61 is reported as (5983,20568,61), but 5983^2 is nowhere near 61*20568^2.
Inside of the comprehension creating unnecessary Double instances on each value of y.
I couldn't find a solution using list comprehensions that didn't have the space blowup. But rewriting using recursion yields a stable memory profile.
diophantine :: Int -> Int -> Int -> Bool
diophantine x y d = x^2 - d*y^2 == 1
minimalsolution :: Int -> (Int, Int, Int)
minimalsolution d = go 2
where
d0 = fromIntegral d
go a =
let y = fromIntegral a
x = round $ sqrt $ (d0*y^2+1) in
if diophantine x y d then
(x, y, d)
else
go (y+1)
For what it is worth I have tested this now after 6 years and this problem does not appear anymore. The memory consumption stays very low with GHC 8.6.5. I assume that this was indeed a problem in the compiler which has been fixed at some point.
Related
--for number divisible by 15 we can get it easily
take 10 [x | x <- [1..] , x `mod` 15 == 0 ]
--but for all how do I use the all option
take 10 [x | x <- [1..] , x `mod` [2..15] == 0 ]
take 10 [x | x <- [1..] , all x `mod` [2..15] == 0 ]
I want to understand how to use all in this particular case.
I have read Haskell documentation but I am new to this language coming from Python so I am unable to figure the logic.
First you can have a function to check if a number is mod by all [2..15].
modByNumbers x ns = all (\n -> x `mod` n == 0) ns
Then you can use it like the mod function:
take 10 [x | x <- [1..] , x `modByNumbers` [2..15] ]
Alternatively, using math, we know that the smallest number divible by all numbers less than n is the product of all of the prime numbers x less than n raised to the floor of the result of logBase x n.
A basic isPrime function:
isPrime n = length [ x | x <- [2..n], n `mod` x == 0] == 1
Using that to get all of the primes less than 15:
p = [fromIntegral x :: Float | x <- [2..15], isPrime x]
-- [2.0,3.0,5.0,7.0,11.0,13.0]
Now we can get the exponents:
e = [fromIntegral (floor $ logBase x 15) :: Float | x <- p']
-- [3.0,2.0,1.0,1.0,1.0,1.0]
If we zip these together.
z = zipWith (**) p e
-- [8.0,9.0,5.0,7.0,11.0,13.0]
And then find the product of these we get the smallest number divisible by all numbers between 2 and 15.
smallest = product z
-- 360360.0
And now to get the rest we just need to multiply that by the numbers from 1 to 15.
map round $ take 10 [smallest * x | x <- [1..15]]
-- [360360,720720,1081080,1441440,1801800,2162160,2522520,2882880,3243240,3603600]
This has the advantage of running substantially faster.
Decompose the problem.
You already know how to take the first 10 elements of a list, so set that aside and forget about it. There are infinitely many numbers divisible by all of [2,15], your remaining task is to list them all.
There are infinitely many natural numbers (unconstrained), and you already know how to list them all ([1..]), so your remaining task is to transform that list into the "sub-list" who's elements are divisible by all of [2,15].
You already know how to transform a list into the "sub-list" satisfying some constraint (predicate :: X -> Bool). You're using a list comprehension in your posted code, but I think the rest of this is going to be easier if you use filter instead. Either way, your remaining task is to represent "is divisible by all of [2,15]" as a predicate..
You already know how to check if a number x is divisible by another number y. Now for something new: you want to abstract that as a predicate on x, and you want to parameterize that predicate by y. I'm sure you could get this part on your own if asked:
divisibleBy :: Int -> (Int -> Bool)
divisibleBy y x = 0 == (x `mod` y)
You already know how to represent [2,15] as [2..15]; we can turn that into a list of predicates using fmap divisibleBy. (Or map, worry about that difference tomorrow.) Your remaining task is to turn a list of predicates into a predicate.
You have a couple of options, but you already found all :: (a -> Bool) -> [a] -> Bool, so I'll suggest all ($ x). (note)
Once you've put all these pieces together into something that works, you'll probably be able to boil it back down into something that looks a little bit like what you first wrote.
Task is to find all two-valued numbers representable as the sum of the sqrt's of two natural numbers.
I try this:
func = [sqrt (x) + sqrt (y) | x <- [10..99], y <- [10..99], sqrt (x) `mod` 1 == 0, sqrt (y) `mod` 1 == 0]
Result:
Unresolved top-level overloading Binding : func
Outstanding context : (Integral b, Floating b)
How can I fix this?
This happens because of a conflict between these two types:
sqrt :: Floating a => a -> a
mod :: Integral a => a -> a -> a
Because you write mod (sqrt x) 1, and sqrt is constrained to return the same type as it takes, the compiler is left trying to find a type for x that simultaneously satisfies the Floating constraint of sqrt and the Integral constraint of mod. There are no types in the base library that satisfy both constraints.
A quick fix is to use mod' :: Real a => a -> a -> a:
import Data.Fixed
func = [sqrt (x) + sqrt (y) | x <- [10..99], y <- [10..99], sqrt (x) `mod'` 1 == 0, sqrt (y) `mod'` 1 == 0]
However, from the error you posted, it looks like you may not be using GHC, and mod' is probably a GHC-ism. In that case you could copy the definition (and the definition of the helper function div') from here.
But I recommend a more involved fix. The key observation is that if x = sqrt y, then x*x = y, so we can avoid calling sqrt at all. Instead of iterating over numbers and checking if they have a clean sqrt, we can iterate over square roots; their squares will definitely have clean square roots. A straightforward application of this refactoring might look like this:
sqrts = takeWhile (\n -> n*n <= 99)
. dropWhile (\n -> n*n < 10)
$ [0..]
func = [x + y | x <- sqrts, y <- sqrts]
Of course, func is a terrible name (it's not even a function!), and sqrts is a constant we could compute ourselves, and is so short we should probably just inline it. So we might then simplify to:
numberSums = [x + y | x <- [4..9], y <- [4..9]]
At this point, I would be wondering whether I really wanted to write this at all, preferring just
numberSums = [8..18]
which, unlike the previous iteration, doesn't have any duplicates. It has lost all of the explanatory power of why this is an interesting constant, though, so you would definitely want a comment.
-- sums of pairs of numbers, each of whose squares lies in the range [10..99]
numberSums = [8..18]
This would be my final version.
Also, although the above definitions were not parameterized by the range to search for perfect squares in, all the proposed refactorings can be applied when that is a parameter; I leave this as a good exercise for the reader to check that they have understood each change.
When Data.List.minimumBy comes across list elements that are equal, it chooses the one that came first, but maximumBy chooses the last one:
> import Data.List
> import Data.Ord
> minimumBy (const (const EQ)) "Hello world!"
'H'
> maximumBy (const (const EQ)) "Hello world!"
'!'
Is this by design or by coincidence? Is there a good reasoning behind this behavior?
Note that taking advantage of such an assumption can make code much more succinct - i.e minimumOn length texts instead of using an explicit tie-breaker such as map snd (minimumOn (\(p, t) -> (length t, p)) (zip [0..] texts))
In the Haskell report there is a comment about min and max:
-- note that (min x y, max x y) = (x,y) or (y,x)
max x y
| x <= y = y
| otherwise = x
min x y
| x <= y = x
| otherwise = y
minimumBy and maximumBy are probably using these or at least trying to stay consistent with them.
I would guess the reason is that you might use min and max to, say, write a sort that involved comparing and swapping pairs (as in the operation in the comment above). Without this property you could lose elements and have others duplicated.
You normally wouldn't be able to observe any difference, since things that compare equal are usually completely identical. But you can imagine having elements with some sort of internal structure that isn't considered by comparisons but that you would not want to be lost during a sort.
Apart from David Fletcher's point (and related to it), I'd reckon this behaviour is to preserve the invariant
minimum l ≡ head (sort l)
maximum l ≡ last (sort l)
also in its generalisation
minimumBy c l ≡ head (sortBy c l)
maximumBy c l ≡ last (sortBy c l)
...even if c behaves pathologically like your const (const EQ). And because sort is stable, that means simply the list will be kept as-is, hence minimum should directly pick the head and maximum the last then.
f x y z = [n | n <- z, n > x + y]
f 1 2 [3,4]
Would x + y be executed only once at first so that the successive calls be replaced by the value 3 instead? Is GHC Haskell optimized up to this job for FP brings us the virtue of referential transparency?
How to trace to prove it?
I don't think the computed value will be reused.
The general problem with this kind of thing is, x + y is cheap, but you could instead have some operation there that produces an utterly vast result, which you probably don't want to keep in memory. Which is a wordy way of saying "this is a time/space tradeoff".
Because of this, it seems GHC tends to not reuse work, in case the lost space doesn't make up for the gained time.
The way to find out for sure is to ask GHC to dump Core when it compiles your code. You can then see precisely what's going to get executed. (Be prepared for it to be very verbose though!) Oh, and make sure you turn on optimisations! (I.e., the -O2 flag.)
If you rephrase your function as
f x y z = let s = x + y in [ n | n <- z, n > s ]
Now s will definitely be executed only once. (I.e., once per call to f. Each time you call f it'll still recompute s.)
Incidentally, if you're interested in saving already-computed results for the whole function, the search term you're looking for is "memoisation".
What will happen can depend on whether you are using ghci vs. ghc and then, if you are compiling the code, what optimization level is being used.
Here is one way to test the evaluations:
import Debug.Trace
f x y z = [n | n <- z, n > tx x + ty y]
where tx = trace "x"
ty = trace "y"
main = print $ f 1 2 [3,4]
With 7.8.3 I get the following results:
ghci: x y x y [4]
ghc (no optimization): x y x y [4]
ghc -O2: x y [4]
It is possible that the addition of the trace calls affects CSE optimization. But this does show that -O2 will hoist x+y out of the loop.
I started learning haskell yesterday and I got stuck on a problem. After a bit of trying different things I thought I'd finally come here and ask how to fix this. Also, feel free to criticize the way I have done things so far so I can know what direction to go. Thanks.
module Main where
main = putStrLn lastPrime
where
lastPrime :: String
lastPrime = show(last(take 10001 primes))
primes :: [Int]
primes = [x| x <- [1..],length [a| a <- [1..lessen(x)], mod x a /= 0] == x - 2]
lessen :: Int -> Int
lessen a = ceiling(sqrt(a))
To fix your type error, you want this:
lessen :: Int -> Int
lessen a = ceiling (sqrt (fromIntegral a))
a has type Int, but sqrt is expecting a floating point type, and the easiest way to convert an integral type to another numeric type is fromIntegral.
In addition to the type error in lessen you have a logic error in primes:
length [a| a <- [1..lessen(x)], mod x a /= 0] == x - 2
You're (rightly) only considering elements up to lessen x. This has the consequence that the list will almost never have exactly x - 2 elements. As a consequence you'll get into an infinite loop when trying to get more than two elements out of that list (because there is no 3rd element for which the condition is true, but haskell doesn't know that so it iterates to infinity trying to find it).
Also note that taking the length of a list is an O(n) operation and there's almost always a better way to achieve what you want.
As a style note, I would recommend defining a separate method isPrime. This will make your code look like this:
module Main where
main = putStrLn lastPrime
where
lastPrime :: String
lastPrime = show(last(take 10001 primes))
isPrime x = length [a| a <- [1..lessen(x)], mod x a /= 0] == x - 2]
primes :: [Int]
primes = [x| x <- [1..], isPrime x]
lessen :: Int -> Int
lessen a = ceiling(sqrt (fromIntegral a))
This IMHO makes the list comprehension much more readable. However it still contains the bug. To get rid of the bug, I'd suggest defining isPrime using a different approach. Going up to lessen x is fine (except you should start from 2 because 1 cleanly divides everything), but instead of building a new list with all the divisors, you should just check whether any of the numbers in the range divides x. For this we can use the higher order function any, so we get this:
isPrime x = not (any (\a -> mod x a == 0) [2 .. lessen x])