Perplexing behaviour when approximating the derivative in haskell - haskell

I have defined a typeclass Differentiable to be implemented by any type which can operate on infinitesimals.
Here is an example:
class Fractional a => Differentiable a where
dif :: (a -> a) -> (a -> a)
difs :: (a -> a) -> [a -> a]
difs = iterate dif
instance Differentiable Double where
dif f x = (f (x + dx) - f(x)) / dx
where dx = 0.000001
func :: Double -> Double
func = exp
I have also defined a simple Double -> Double function to differentiate.
But when I test this in the ghc this happens:
... $ ghci
GHCi, version 8.8.4: https://www.haskell.org/ghc/ :? for help
Prelude> :l testing
[1 of 1] Compiling Main ( testing.hs, interpreted )
Ok, one module loaded.
*Main> :t func
func :: Double -> Double
*Main> derivatives = difs func
*Main> :t derivatives
derivatives :: [Double -> Double]
*Main> terms = map (\f -> f 0) derivatives
*Main> :t terms
terms :: [Double]
*Main> take 5 terms
[1.0,1.0000004999621837,1.000088900582341,-222.0446049250313,4.440892098500626e8]
*Main>
The approximations to the nth derivative of e^x|x=0 are:
[1.0,1.0000004999621837,1.000088900582341,-222.0446049250313,4.440892098500626e8]
The first and 2nd derivatives are perfectly reasonable approximations given the setup, but suddenly, the third derivative of func at 0 is... -222.0446049250313! HOW!!?

The method you're using here is a finite difference method of 1st-order accuracy.
Layman's translation: it works, but is pretty rubbish numerically speaking. Specifically, because it's only 1st-order accurate, you need those really small steps to get good accuracy even with exact-real-arithmetic. You did choose a small step size so that's fine, but small step size brings in another problem: rounding errors. You need to take the difference f (x+δx) - f x with small δx, meaning the difference is small whereas the individual values may be large. That always brings up the floating-point inaccuracy – consider for example
Prelude> (1 + pi*1e-13) - 1
3.141931159689193e-13
That might not actually hurt that much, but since you then need to divide by δx you boost up the error.
This issue just gets worse/compounded as you go to the higher derivatives, because now each of the f' x and f' (x+δx) has already an (non-identical!) boosted error on it, so taking the difference and boosting again is a clear recipe for disaster.
The simplest way to remediate the problem is to switch to a 2nd-order accurate method, the obvious being central difference. Then you can make the step a lot bigger, and thus largely avoid rounding issues:
Prelude> let dif f x = (f (x + δx) - f(x - δx)) / (2*δx) where δx = 1e-3
Prelude> take 8 $ ($0) <$> iterate dif exp
[1.0,1.0000001666666813,1.0000003333454632,1.0000004990740052,0.9999917560676863,0.9957312752106873,8.673617379884035,7806.255641895632]
You see the first couple of derivatives are good now, but then eventually it also becomes unstable – and this will happen with any FD method as you iterate it. But that's anyway not really a good approach: note that every evaluation of the n-th derivative requires 2 evaluations of the n−1-th. So, the complexity is exponential in the derivative degree.
A better approach to approximate the n-th derivative of an opaque function is to fit an n-th order polynomial to it and differentiate this symbolically/automatically. Or, if the function is not opaque, differentiate itself symbolically/automatically.

tl;dr: the dx denominator gets small exponentially quickly, which means that even small errors in the numerator get blown out of proportion.
Let's do some equational reasoning on the first "bad" approximation, the third derivative.
dif (dif (dif exp))
= { definition of dif }
dif (dif (\x -> (exp (x+dx) - exp x)/dx))
= { definition of dif }
dif (\y -> ((\x -> (exp (x+dx) - exp x)/dx) (y+dx)
- (\x -> (exp (x+dx) - exp x)/dx) y
)/dx)
= { questionable algebra }
dif (\y -> (exp (y + 2*dx) - 2*exp (y + dx) + exp y)/dx^2)
= { alpha }
dif (\x -> (exp (x + 2*dx) - 2*exp (x + dx) + exp x)/dx^2)
= { definition of dif and questionable algebra }
\x -> (exp (x + 3*dx) - 3*exp (x + 2*dx) + 3*exp (x + dx) - exp x)/dx^3
Hopefully by now you can see the pattern we're getting into: as we take more and more derivatives, the error in the numerator gets worse (because we are computing exp farther and farther away from the original point, x + 3*dx is three times as far away e.g.) while the sensitivity to error in the denominator gets higher (because we are computing dx^n for the nth derivative). By the third derivative, these two factors become untenable:
> exp (3*dx) - 3*exp (2*dx) + 3*exp (dx) - exp 0
-4.440892098500626e-16
> dx^3
9.999999999999999e-19
So you can see that, although the error in the numerator is only about 5e-16, the sensitivity to error in the denominator is so high that you start to see nonsensical answers.

Related

Unresolved top level overloading

Task is to find all two-valued numbers representable as the sum of the sqrt's of two natural numbers.
I try this:
func = [sqrt (x) + sqrt (y) | x <- [10..99], y <- [10..99], sqrt (x) `mod` 1 == 0, sqrt (y) `mod` 1 == 0]
Result:
Unresolved top-level overloading Binding : func
Outstanding context : (Integral b, Floating b)
How can I fix this?
This happens because of a conflict between these two types:
sqrt :: Floating a => a -> a
mod :: Integral a => a -> a -> a
Because you write mod (sqrt x) 1, and sqrt is constrained to return the same type as it takes, the compiler is left trying to find a type for x that simultaneously satisfies the Floating constraint of sqrt and the Integral constraint of mod. There are no types in the base library that satisfy both constraints.
A quick fix is to use mod' :: Real a => a -> a -> a:
import Data.Fixed
func = [sqrt (x) + sqrt (y) | x <- [10..99], y <- [10..99], sqrt (x) `mod'` 1 == 0, sqrt (y) `mod'` 1 == 0]
However, from the error you posted, it looks like you may not be using GHC, and mod' is probably a GHC-ism. In that case you could copy the definition (and the definition of the helper function div') from here.
But I recommend a more involved fix. The key observation is that if x = sqrt y, then x*x = y, so we can avoid calling sqrt at all. Instead of iterating over numbers and checking if they have a clean sqrt, we can iterate over square roots; their squares will definitely have clean square roots. A straightforward application of this refactoring might look like this:
sqrts = takeWhile (\n -> n*n <= 99)
. dropWhile (\n -> n*n < 10)
$ [0..]
func = [x + y | x <- sqrts, y <- sqrts]
Of course, func is a terrible name (it's not even a function!), and sqrts is a constant we could compute ourselves, and is so short we should probably just inline it. So we might then simplify to:
numberSums = [x + y | x <- [4..9], y <- [4..9]]
At this point, I would be wondering whether I really wanted to write this at all, preferring just
numberSums = [8..18]
which, unlike the previous iteration, doesn't have any duplicates. It has lost all of the explanatory power of why this is an interesting constant, though, so you would definitely want a comment.
-- sums of pairs of numbers, each of whose squares lies in the range [10..99]
numberSums = [8..18]
This would be my final version.
Also, although the above definitions were not parameterized by the range to search for perfect squares in, all the proposed refactorings can be applied when that is a parameter; I leave this as a good exercise for the reader to check that they have understood each change.

No instance for (Fractional Int) arising from a use of `area'

I'm new to Haskell and I'm writing a program that calculates the limit of a function. So given two lists a and b, a delta dx = 0.001, and the limits of integration l and r, I want to recursively compute the area under the curve with equation:
a1(x)^b1 + a2(x)^b2 + ... + an(x)bn where x is all the values between l an r with an increment of dx between each value. The technical part isn't that important I guess but it helps to read the code:
import Text.Printf (printf)
-- This function should return a list [area].
solve :: Int -> Int -> [Int] -> [Int] -> [Double]
solve l r x y = [area l r x y]
area l r a b = if (l < r)
then (calc l a b) * 0.001 + (area (l + 1) r a b)
else (calc r a b) * 0.001
calc n (a:arest) (b:brest) = (fromIntegral(n) ^^ b) * fromIntegral(a) + (calc n arest brest)
calc n [] [] = 0
--Input/Output.
main :: IO ()
main = getContents >>= mapM_ (printf "%.1f\n"). (\[a, b, [l, r]] -> solve l r a b). map (map read. words). lines
I get no error with the above code but as soon as I change area (l + 1) r a b to area (l + 0.001) r a b I get the following error message:
No instance for (Fractional Int) arising from a use of `area'
I tried making a new class and having a be an abstract type but that didn't work, any other ideas?
So the problem is that Int is not a Fractional type. In other words, it does not have a value called 0.001 [note 1], but you have requested Haskell to give you such a value in your code.
You are making this request because 0.001 is fed to the (+) function with another argument (in this case l) which is of type Int. This is a problem because the function has type (+) :: (Num a) => a -> a -> a: in other words, there are a lot of different functions (+) all having the type a -> a -> a; one of these functions exists for every type a in the Num type class.
Since we know that one argument to the function is an Int, it follows that we're using the specific function (+) :: Int -> Int -> Int. That is why l + 0.001 gets weird.
As for solving the problem: You probably wanted l and r to be of type Double (they're left and right bounds on where a number can be?) but if you're sure that they must be Ints then you probably meant to write fromIntegral l + 0.001.
Side note on style: parentheses in Haskell are always just grouping/precedence, functions are higher precedence than operators which are higher precedence than special forms (let, case, if, do), and function application is always left-associative or "greedy nom": a function eats whatever is immediately in front of it. What you have written:
(fromIntegral(n) ^^ b) * fromIntegral(a) + (calc n arest brest)
is probably better written as:
fromIntegral a * fromIntegral n ^^ b + calc n arest brest
The parentheses around calc are not necessary (because operators like + have lower precedence than function applications), nor are the parentheses around n and a (because those sub-expressions are indivisible chunks; fromIntegral(n) is identical to fromIntegral (n) is identical to fromIntegral n).
As #dfeuer mentions below: secretly, when you write 0.001 it does not have a definite type; rather it is translated to fromRational 0.001 internally, where the latter 0.001 is a definite value of the definite type Rational, just as when you write 4 it is translated to fromInteger 4 where the latter 4 is a definite value of the definite type Integer. The problem is really that there is no fromRational function for Int, because Int is not part of the Fractional typeclass which defines fromRational. And it's not part of that typeclass because the language designers preferred an error to a silent rounding/dropping of a fraction.

Haskell - Instance of Fractional Int required for definition of filename?

I'm an amateur at Haskell, trying to run through an infinite list of approximations to the square root of "x", where "acc" represents the generation this step is on. However, when I run the code below, I get the underlying error.
as' x acc = ( last(take (acc-1) (as' x (acc-1)))
+ (acc / last(take (acc-1) (as' x (acc-1)))) ) / 2 : as' x (acc+1)
ERROR "a5.hs":34 - Instance of Fractional Int required for definition of as'
Also, when I try applying this type code, I get an error:
as' :: Float -> Float -> Float
Type error in application
*** Expression : (last (take (acc - 1) (as' x (acc - 1))) + acc / last (take (acc - 1) (as' x (acc - 1)))) / 2 : as' x (acc + 1)
*** Term : as' x (acc + 1)
*** Type : Float
*** Does not match : [a]
EDIT: To offer you some clarity, I want to use this function in the context of a list.
e.g. as x = [1, as' x 2].
The idea is that this will accumulate an infinite list, since as' will recursively call itself. Hence why I felt I could operate on a list here.
Can anyone please offer me some clarity?
The type signature of take is
take :: Int -> [a] -> [a]
Here's how you are using take:
take (acc-1) (as' x (acc-1))
So we can conclude that
(acc-1) :: Int -- first parameter to `take`
acc :: Int -- therefore
(as' x (acc-1)) :: [a] -- second parameter to `take`, we don't know what `a` is
But your code says
as' :: Float -> Float -> Float
as' x acc = ...
From which we deduce
x :: Float -- first parameter to `as'`
acc :: Float -- second parameter to `as'`
(as' x (acc-1)) :: Float -- result of `as'`
Which leads to a couple of contradictions:
acc cannot be an Int and a Float at the same time
(as' x (acc-1)) cannot be an [a] and a Float at the same time --- this is what the second error message is trying to tell you
Ultimately, you are trying to use take on something that is not a list. I'm not sure what you are trying to do.
You probably intended to have the signature
as' :: Float -> Int -> [Float]
That should (I've not tested it) fix the type errors above, but still leaves a more fundamental problem: whenever you compute the nth element of the list, you compute the *n-1*th element of the list anew twice (and so on, back to the start of the list: exponential growth of recalculation), even though presumably this element has already been computed. There is no sharing going on.
e.g. consider
as' x acc = ( prev + (acc / prev) ) / 2 : as' x (acc+1)
where prev = last(take (acc-1) (as' x (acc-1)))
This is still inefficient: you still recompute previous elements of the list. But now you only recompute all previous elements once when computing the next element.
(It would also be remiss of me not to point out that last(take (acc-1) (as' x (acc-1))) can be simplified to (as' x (acc-1)) !! (acc-2).)
The usual way to generate an infinite list where each element depends only on the previous element is to use iterate.
The complication is that you have each element depending on an accumulator as well as depending on the previous element. We will get round that by incorporating the accumulator into each element of the list. When we are done we will throw away the accumulators to produce our final infinite list.
approxRoots :: Float -> [Float]
approxRoots x = map fst $ iterate next (x, 1)
-- I don't know what your initial approximation should be
-- I've put `x` but that's probably wrong
where next (prev, acc) = (prev + acc / prev, acc + 1)
-- First element of each pair is the approximation,
-- second element of each pair is the "accumulator" (actually an index)
-- I've probably transcribed your formula wrongly
dave4420's answer is already very nice, I just want to share how you can get the most from the error message which the compiler gave you. Here is it again:
*** Expression : (last (take (acc - 1) (as' x (acc - 1))) + acc / last (take (acc - 1) (as' x (acc - 1)))) / 2 : as' x (acc + 1)
*** Term : as' x (acc + 1)
*** Type : Float
*** Does not match : [a]
This means that the as' x (acc + 1) part in the long expression was expected to yield a list, but it actually gives a Float value.
Why does the compiler expect it to be a list? Well, let's see where the term is used in the expression:
(last .... ) / 2 : as' x (acc + 1)
I.e., it's used as the second argument to the (:) function, and the compiler knows that the second argument to this function has to be a list (the compiler knows that the signature of (:) is a -> [a] -> [a] though it doesn't mention that part in the error message).
Why is it actually a Float? Since you didn't provide a function signature, the compiler deduced it for you and actually printed it as well:
as' :: Float -> Float -> Float
So the compiler determined that as' takes two Float values and yields a Float value. I don't know from the top of my head why it did that.
My advice is to start debugging this issue by explicitely writing down a function signature yourself. Doing so will cause a different error message which is closer to the cause of the mismatch between your expectation and the actual code.

Calling functions prefix vs. infix in Haskell

I've been wanting to learn Haskell, so recently I started working through the ProjectEuler problems. While writing the following factoring code I noticed that calling (/ n) returns a Float while (n `div`) returns an Int. I thought that infix notation was simply syntactic sugar in Haskell? Could someone explain what is going on? I would also appreciate any comments / suggestions / improvements, thank you.
import Data.List (sort)
factor :: Int -> [Int]
factor 0 = [1..]
factor n =
let f1 = [f | f <- [1..limit], n `mod` f == 0]
where limit = ceiling $ sqrt $ fromIntegral n
f2 = map (n `div`) f1 --vs. map (/ n) f1
in sort $ f1 ++ f2
div and / are two different functions:
/ is defined in class Fractional and it's meaning is an inverse operation to multiplication.
div is defined in class Integral and it's meaning is division of integers with truncation toward negative infinity.
You're right, infix notation is just a syntactic sugar. The expression x / y is the same as (/) x y, as well as div x y is the same as x `div` y.
There's nothing special going on. The div function is part of the Integral class and is being more specifically inferred as Int, given your explicit type signature. The / operator is part of the Fractional class. These are two different functions, one is not syntactic sugar for another!

Project euler problem 3 in haskell

I'm new in Haskell and try to solve 3 problem from http://projecteuler.net/.
The prime factors of 13195 are 5, 7, 13 and 29.
What is the largest prime factor of the number 600851475143 ?
My solution:
import Data.List
getD :: Int -> Int
getD x =
-- find deviders
let deriveList = filter (\y -> (x `mod` y) == 0) [1 .. x]
filteredList = filter isSimpleNumber deriveList
in maximum filteredList
-- Check is nmber simple
isSimpleNumber :: Int -> Bool
isSimpleNumber x = let deriveList = map (\y -> (x `mod` y)) [1 .. x]
filterLength = length ( filter (\z -> z == 0) deriveList)
in
case filterLength of
2 -> True
_ -> False
I try to run for example:
getD 13195
> 29
But when i try:
getD 600851475143
I get error Exception: Prelude.maximum: empty list Why?
Thank you #Barry Brown, I think i must use:
getD :: Integer -> Integer
But i get error:
Couldn't match expected type `Int' with actual type `Integer'
Expected type: [Int]
Actual type: [Integer]
In the second argument of `filter', namely `deriveList'
In the expression: filter isSimpleNumber deriveList
Thank you.
Your type signature limits the integer values to about 2^29. Try changing Int to Integer.
Edit:
I see that you already realised that you need to use Integer instead of Int. You need to change the types of both getD and isSimpleNumber otherwise you will get a type mismatch.
Also in general, if you are having trouble with types, simply remove the type declarations and let Haskell tell you the correct types.
Main> :t getD
getD :: Integral a => a -> a
Main> :t isSimpleNumber
isSimpleNumber :: Integral a => a -> Bool
After you found the error, may I point out that your solution is quite verbose? In this case a very simple implementation using brute force is good enough:
getD n = getD' n 2 where
getD' n f | n == f = f
| n `mod` f == 0 = getD' (n `div` f) f
| otherwise = getD' n (succ f)
this question is easy enough for brute-force solution, but it is a bad idea to do so because the whole idea of project euler is problems you need to really think of to solve (see end of answer)
so here are some of your program's flaws:
first, use rem instead of mod. it is more efficient.
some mathematical thinking should have told you that you don't need to check all numbers from 1 to x in the isprime function and the getD function, but checking all numbers from the squareroot to one (or reversed) should be sufficient. note that in getD you will actually need to filter numbers between x and the square root, because you search for the biggest one.
why do you use the maximum function in getD? you know the list is monotonically growing, so you may as well get the last one.
despite you only need the biggest divisor (which is prime) you compute the divisors list from small to big making the computer check for each value if it is a divisor or not although discarding the result once a bigger divisor is found. it should be fixed by filtering the list of numbers from x to 1, not from 1 to x. this will cause the computer to check divisibility (how should I say that?) for the biggest possible divisor, not throwing to the trash the knowledge of previous checks. note that this optimization takes effect only if the previous point is optimized, because otherwise the computer will compute all divisors anyway.
with the previous points mixed, you should have filtered all numbers [x,x-1 .. squareroot x] and taken the first.
you don't use an efficient isPrime function. if I were you, I would have searched for an isprime library function, which is guaranteed to be efficient.
and there are more..
with this kind of code you will never be able to solve harder project euler problems. they are designed to need extra thinking about the problem (for instance noticing you don't have to check numbers greater from the square root) and writing fast and efficient code. this is the purpose of project euler; being smart about programming. so don't skip it.

Resources