Lazy evaluation of terms in an infinite list in Haskell - haskell

I am curious about the runtime performance of an infinite list like
the one below:
fibs = 1 : 1 : zipWith (+) fibs (tail fibs)
This will create an infinite list of the fibonacci sequence.
My question is that if I do the following:
takeWhile (<5) fibs
how many times does fibs evaluate each term in the list? It seems
that since takeWhile checks the predicate function for each item in
the list, the fibs list will evaluate each term multiple times. The
first 2 terms are given for free. When takeWhile wants to evaluate
(<5) on the 3rd element, we will get:
1 : 1 : zipWith (+) [(1, 1), (1)] => 1 : 1 : 3
Now, once takeWhile wants to evaluate (<5) on the 4th element: the
recursive nature of fibs will construct the list again like the
following:
1 : 1 : zipWith (+) [(1, 2), (2, 3)] => 1 : 1 : 3 : 5
It would seem that the 3rd element needs to be computed again when we
want to evaluate the value of the 4th element. Furthermore, if the
predicate in takeWhile is large, it would indicate the function is
doing more work that is needed since it is evaluating each preceding
element in the list multiple times. Is my analysis here correct or is
Haskell doing some caching to prevent multiple evaluations here?

This is a self-referential, lazy data structure, where "later" parts of the structure refer to earlier parts by name.
Initially, the structure is just a computation with unevaluated pointers back to itself. As it is unfolded, values are created in the structure. Later references to already-computed parts of the structure are able to find the value already there waiting for them. No need to re-evaluate the pieces, and no extra work to do!
The structure in memory begins as just an unevaluated pointer. Once we look at the first value, it looks like this:
> take 2 fibs
(a pointer to a cons cell, pointing at '1', and a tail holding the second '1', and a pointer to a function that holds references back to fibs, and the tail of fibs.
Evaluating one more step expands the structure, and slides the references along:
And so we go unfolding the structure, each time yielding a new unevaluated tail, which is a closure holding references back to 1st and 2nd elements of the last step. This process can continue infinitely :)
And because we're referring to prior values by name, GHC happily retains them in memory for us, so each item is evaluated only once.

Illustration:
module TraceFibs where
import Debug.Trace
fibs :: [Integer]
fibs = 0 : 1 : zipWith tadd fibs (tail fibs)
where
tadd x y = let s = x+y
in trace ("Adding " ++ show x ++ " and " ++ show y
++ "to obtain " ++ show s)
s
Which produces
*TraceFibs> fibs !! 5
Adding 0 and 1 to obtain 1
Adding 1 and 1 to obtain 2
Adding 1 and 2 to obtain 3
Adding 2 and 3 to obtain 5
5
*TraceFibs> fibs !! 5
5
*TraceFibs> fibs !! 6
Adding 3 and 5 to obtain 8
8
*TraceFibs> fibs !! 16
Adding 5 and 8 to obtain 13
Adding 8 and 13 to obtain 21
Adding 13 and 21 to obtain 34
Adding 21 and 34 to obtain 55
Adding 34 and 55 to obtain 89
Adding 55 and 89 to obtain 144
Adding 89 and 144 to obtain 233
Adding 144 and 233 to obtain 377
Adding 233 and 377 to obtain 610
Adding 377 and 610 to obtain 987
987
*TraceFibs>

When something is evaluated in Haskell, it stays evaluated, as long as it's referenced by the same name1.
In the following code, the list l is only evaluated once (which might be obvious):
let l = [1..10]
print l
print l -- None of the elements of the list are recomputed
Even if something is partially evaluated, that part stays evaluated:
let l = [1..10]
print $ take 5 l -- Evaluates l to [1, 2, 3, 4, 5, _]
print l -- 1 to 5 is already evaluated; only evaluates 6..10
In your example, when an element of the fibs list is evaluated, it stays evaluated. Since the arguments to zipWith reference the actual fibs list, it means that the zipping expression will use the already partially computed fibs list when computing the next elements in the list. This means that no element is evaluated twice.
1This is of course not strictly required by the language semantics, but in practice this is always the case.

Think of it this way. The variable fib is a pointer to a lazy value. (You can think of a lazy value underneath as a data structure like (not real syntax) Lazy a = IORef (Unevaluated (IO a) | Evaluated a); i.e. it starts out as unevaluated with a thunk; then when it is evaluated it "changes" to something that remembers the value.) Because the recursive expression uses the variable fib, they have a pointer to the same lazy value (they "share" the data structure). The first time someone evaluates fib, it runs the thunk to get the value and that value is remembered. And because the recursive expression points to the same lazy data structure, when they evaluate it, they will see the evaluated value already. As they traverse the lazy "infinite list", there will only be one "partial list" in memory; zipWith will have two pointers to "lists" which are simply pointers to previous members of the same "list", due to the fact that it started with pointers to the same list.
Note that this is not really "memoizing"; it's just a consequence of referring to the same variable. There is generally no "memoizing" of function results (the following will be inefficient):
fibs () = 0 : 1 : zipWith tadd (fibs ()) (tail (fibs ()))

Related

Understanding the Limitations of Lazy Evaluation (Sieve of Eratosthenes)

In the Haskell Wiki article on prime numbers, the following implementation of the Sieve of Eratosthenes is described:
primes = 2 : 3 : minus [5,7..] (unionAll [[p*p, p*p+2*p..] | p <- tail primes])
When doing...
primes !! 2
... how does Haskell recognize in this specific situation not to try all p's in the tail of primes (a.k.a [3..]), but instead only does a set minus with 3?
In other words: how does Haskell know that any of the other p's (or multiples thereof) will not match 5, which is the eventual answer. Is there a good rule of thumb to know when the compiler is smart enough to handle infinite cases?
(!!) only demands that primes be evaluated enough to find out that there are at least 3 elements, and what the third element is. To get that third element, we need to start evaluating the call to minus.
minus assumes that both its arguments are sorted. This is described in the docs, and is satisfied in the definition of primes. The first comparison minus performs is between 5 and 9, and this shows that 5 is the first element of the result. In the definition of minus this is the case LT -> x : loop xs (y:ys).
(!!) now has the third element of primes, so evaluation does not continue in primes or minus or unionAll. This back-and-forth between evaluation of subexpressions and pattern matching in the outer expressions is lazy evaluation.
Actually, the crux is in the implementation of unionAll. minus just pulls elements from its right argument one by one unawares (it assumes both its arguments are non-decreasing of course).
First, let's re-write it as
primes = 2 : ps
ps = 3 : t
t = minus [5,7..] (unionAll [[p*p, p*p+2*p..] | p <- ps])
-- primes !! 2 == ps !! 1 == head t
= minus [5,7..] (unionAll [[p*p, p*p+2*p..] | p <- (3 : t)])
= minus [5,7..] (unionAll ([9, 15..] : [[p*p, p*p+2*p..] | p <- t]))
Now unionAll is smart: it relies on the assumption (here, fact) that in unionAll xs it holds that (map head xs) are non-decreasing.
As such, it knows it does not have to compare 9 with anything! So it just produces it unconditionally (you can consult its definition to check it for yourself):
= minus [5,7..]
(9 : union [15, 21..] (unionAll ........))
Thus minus has something to compare the 5 and the 7 with, and produces:
= 5 : 7 : minus [9,11..]
(9 : union [15, 21..] (unionAll ........))
All this from knowing just the first odd prime, 3.

Why doesn't product [0..] evaluate to 0 "instantly"?

I am trying to understand laziness. Because 0 multiplied with any number is 0, shouldn't product [0..] evaluate to 0? I tried also foldl (*) 1 [0..], and to define my own product as
myProduct 0 _ = 0
myProduct _ 0 = 0
myProduct a b = a*b
Why doesn't the fold stop as soon as a 0 is found?
Because the multiply operator doesn't know it's getting chained, and the fold function doesn't know the multiply operator's particular behaviour for any argument. With that combination, it needs to exhaust the list to finish the fold. In fact, for this reason foldl doesn't work at all on infinite lists. foldr does, because it can expand the function from the head of the list.
foldl (*) 1 [0..] -> (((..(((1*0)*1)*2)*3....)*inf
The outermost multiplication in the foldl case can never be found, because the list is infinite. It therefore cannot follow the chain to conclude the result is zero. It can, and does, calculate the product along the list, and that product happens to stay zero, but it will not terminate. If you use scanl instead you can see these intermediate products.
foldr (*) 1 [0..] -> 0*(1*(2*(3*((...((inf*1)))...)))
The outermost multiplication in the foldr case is found immediately, because the rest of the list is in fact left as a lazy thunk. It only runs one step:
foldr (*) 1 [0..] -> 0*(foldr (*) 1 [1..])
So because your custom multiplication operator myProduct is not strict in the second argument if the first argument is zero, foldr myProduct 1 [0..] can terminate.
As a side note, the prelude product function is restricted to finite lists (and may be implemented with foldl). Even if it used foldr, it probably would not shortcut because the standard multiply operator is strict; doing otherwise would be computationally expensive in the common case where the products are neither zero nor chained.
-- sum and product compute the sum or product of a finite list of numbers.
sum, product :: (Num a) => [a] -> a
sum = foldl (+) 0
product = foldl (*) 1
In addition, there's a reason it does not use foldr; as we could see in the expansions and scanl function, the left folds can compute as they consume the list. The right fold, if the operator does not shortcut, needs to build an expression as large as the list itself to even begin computation. This difference is because it's the innermost expression that starts the computation in the strict case, but the outermost expression that produces the result, allowing the lazy case. Lazy vs. non-strict in the Haskell wiki might explain better than I can, and even mentions that pattern matching, which you used to describe the shortcut in myProduct, can be strict.
If you switch the first two lines:
myProduct _ 0 = 0
myProduct 0 _ = 0
myProduct a b = a*b
the second argument will always be evaluated before the first one and the infinite foldr won't work anymore.
Since its impossible to define a myProduct that works lazily for both arguments (not evaluating the second if the first is 0 and not evaluating the first if the second is 0) maybe we are better off with having * always evaluate both its arguments.
You can have it thusly:
myproduct xs = foldr op id xs 1
where
op x r acc = if x==0 then 0 else acc `seq` r (acc*x)
This is a right fold that multiplies the numbers from the left, operating in constant space, and stops as soon as a 0 is encountered.

Haskell length + map explanation?

I'm playing around with Haskell since I'm learning the language, and I just found something I don't understand and I can't find an explanation. If I try to run this code:
map (`div` 0) [1,2,3,4]
I get a divide by 0 exception, which is expected.
But if I run this code:
length (map (`div` 0) [1,2,3,4])
I get 4!
I'd like to know why I don't get the divide by 0 exception when I do the mapping inside the length function!
The map and length functions can be defined this way:
map :: (a -> b) -> [a] -> [b]
map _ [] = []
map f (x:xs) = f x : map f xs
length :: [a] -> Int
length [] = 0
length (_:xs) = 1 + length xs
Now, let's work out by hand why your second example works the way it does. It goes like this:
length (map (`div` 0) (1:2:3:4:[]))
= length (1 `div` 0 : map (`div` 0) (2:3:4:[])) -- second map equation
= 1 + (length (map (`div` 0) (2:3:4:[]))) -- second length equation
= 1 + (length (2 `div` 0 : map (`div` 0) (3:4:[]))) -- second map equation
.
.
.
= 1 + (1 + (1 + (1 + length (map (`div` 0) []))))) -- second length equation
= 1 + (1 + (1 + (1 + length [])))) -- first map equation
= 1 + (1 + (1 + (1 + 0)))) -- first length equation
= 4 -- arithmetic
What's the trick here? In Haskell the process of evaluating an expression is called forcing it. Forcing an expression performs the minimum work necessary in order to figure out the outermost data constructor of the result. As part of this, subexpressions will be forced only as necessary to achieve the goal.
So in this example, the outermost expression we're forcing is an application of the length function. The definition of length has two cases, one which uses the [] list constructor and the other which uses the (:) constructor, so to apply length we need to figure out which of these two cases to apply to the argument. Since the argument doesn't have either constructor in its outermost position, we have to force it to find out. That's what happens in the step between the first and the second line of the derivation above; we force the map subexpression by looking at its arguments and choosing the second map equation.
But after that point, we have all the information we need to determine which of the two equations for length applies, so we go by the "outermost first" rule and apply the appropriate length equation. In this case, this discards the subexpression that contains the division by zero, which means that subexpression will never be forced, and the error will never be triggered.
In your first example, however, when you type the expression into GHCI, you're implicitly asking the interpreter to print its result. This requires it to force the spine of the list to access its elements, and force the elements themselves to print them. So the division by zero error happens when you force the first element of the list.
EDIT: Let me point out one nuance you may have not noticed. When we try your first example in GHCI, this is the result of the session:
*Main> map (`div` 0) [1,2,3,4]
[*** Exception: divide by zero
See that lonely opening square bracket at the beginning of the second line? That's the opening bracket for the list that was being printed, before the divide by zero error happened. Likewise, notice what happens in this example:
*Main> map (20 `div`) [1,2,0,4]
[20,10,*** Exception: divide by zero
The first two elements of the result list, and even the comma separating the second element from the third, print successfully because Haskell doesn't attempt to compute the third list element until it needs to be printed.
If you type the map expression into the interpreter, it will evaluate it and then print the resulting list. In order to do that all elements of the resulting list need to be evaluated because they will be part of the string that is displayed.
However when the interpreter evaluates the length expression, it only needs to look at the structure of the resulting list. It does not have to look at the actual elements inside the list. So since Haskell, being a lazy language, only evaluates what it has to, that means that the elements will not be evaluated and thus no exception is thrown.
This is some good old Haskell lazy evaluation! If Haskell doesn't have to compute something, it wont. In this case, you are calling map on a list of length 4. As far as Haskell is concerned, applying map to any list will return a list of the same size, regardless of what operation you are applying. Therefore, Haskell simply tells you the length is 4, without actually dividing anything by 0.

Fibonacci numbers with initial two values as parameters

I have been trying to make a infinite fibonacci list producing function that can take first 2 values as parameters.
Without specifying the first two values it is possible like this
fib = 1 : 1 : zipWith (+) fib (tail fib)
Suppose I wanted to start the fibonacci sequence with 5 and 6 instead of 1,1 or 0,1 then I will have to change the above code. But when trying to make a lazy list generator in which I can specify the first 2 values of fibonacci sequence I am stumped. I came up with this but that didn't work.
fib a b = a : b : zipWith (+) fib (tail fib)
The problem is obvious. I am trying to convert the use of list in the hard-coded one. How can I solve that?
How about
fib a b = fibs where fibs = a : b : zipWith (+) fibs (tail fibs)
? Use the same method, but with your parameters in scope.
I should add that, in case you are tempted by
fib a b = a : b : zipWith (+) (fib a b) (tail (fib a b)) -- worth trying?
the where fibs version ensures that only one infinite stream is generated. The latter risks generating a fresh stream for each recursive invocation of fib. The compiler might be clever enough to spot the common subexpression, but it is not wise to rely on such luck. Try both versions in ghci and see how long it takes to compute the 1000th element.
The simplest way to do that is:
fib a b = a: fib b (a+b)
This stems from the inductive definition of the Fibonacci series: suppose we have a function that can produce a stream of Fibonacci numbers from Fi onwards, given Fi and Fi+1. What could that function look like? Well, Fi is given, and the rest of the stream can be computed using this function to produce a stream of Fibonacci numbers from Fi+1 onwards, if we can provide Fi+1 and Fi+2. Fi+1 is given, so we only need to work out Fi+2. The definition of series gives us Fi+2=Fi+Fi+1, so, there.

Haskell script running out of space

I'm using project Euler to teach myself Haskell, and I'm having some trouble reasoning about how my code is being executed by haskell. The second problem has me computing the sum of all even Fibonacci numbers up to 4 million. My script looks like this:
fibs :: [Integer]
fibs = 1 : 2 : [ a+b | (a,b) <- zip fibs (tail fibs)]
evens :: Integer -> Integer -> Integer
evens x sum | (even x) = x + sum
| otherwise = sum
main = do
print (foldr evens 0 (take 4000000 fibs))
Hugs gives the error "Garbage collection fails to reclaim sufficient space", which I assume means that the list entries are not released as they are consumed by foldr.
What do I need to do to fix this? I tried writing a tail-recursive (I think) version that used accumulators, but couldn't get that to work either.
Firstly, you shouldn't use hugs. It is a toy for teaching purposes only.
GHC, however, is a fast, multicore-ready optimizing compiler for Haskell. Get it here. In particular, it does strictness analysis, and compiles to native code.
The main thing that stands out about your code is the use of foldr on a very large list. Probably you want a tail recursive loop. Like so:
import Data.List
fibs = 0 : 1 : zipWith (+) fibs (tail fibs)
evens x sum | even x = x + sum
| otherwise = sum
-- sum of even fibs in first 4M fibs
main = print (foldl' evens 0 (take 4000000 fibs))
Besides all this, the first 4M even fibs will use a fair amount of space, so it'll take a while.
Here's the sum of the first 400k even fibs, to save you some time (21s). :-)
A number of observations / hints:
the x + sums from even aren't getting evaluated until the very end
You're taking the first 4,000,000 fibs, not the fibs up to 4,000,000
There is an easier way to do this
Edit in response to comment
I'm not going to tell you what the easier way is, since that's the fun of Project Euler problems. But I will ask you a bunch of questions:
How many even fibs can you have in a row?
How long can you go without an even fib?
If you sum up all the even fibs and all the odd fibs (do this by hand), what do you notice about the sums?
You understood the problem wrong. The actual problem wants you to sum all the even Fibonacci numbers such that the Fibonacci number itself doesn't exceed 4 million (which happens to be only the first 33 Fibonacci numbers).
You are evaluating four million elements of fibs. Those numbers grow exponentially. I don't know how many bytes are required to represent the millionth Fibonacci number; just the one-thousandth Fibonacci number has 211 decimal digits, so that's going to take 22 32-bit words just to hold the digits, never mind whatever overhead gmp imposes. And these grow exponentially.
Exercise: calculuate the amount of memory needed to hold four million Fibonacci numbers.
have a look at the Prelude functions takeWhile, filter, even, and sum
takeWhile (<40) [0..]
filter even $ takeWhile (<40) [0..]
put 'em together:
ans = sum $ filter even $ takeWhile (< 4* 10^6) fibs

Resources