I've spent a few minutes debugging a problem that tracked down to "Linear" truncating values that are close to zero when using "Linear.normalize". Specifically, I was taking the cross product of very small triangles and normalizing the result, which, surprisingly, behaved wrongly until I noticed what was wrong and multiplied the cross product by 10000.
Why is that even necessary? How can I get rid of that behavior?
Edit: just for fun, here is a video of the bug. Notice that the sphere loses the color when the number of triangles approximating it is big enough? Yes, good luck debugging that...!
Looking at the source for normalize, you'll see that it's defined as
-- | Normalize a 'Metric' functor to have unit 'norm'. This function
-- does not change the functor if its 'norm' is 0 or 1.
normalize :: (Floating a, Metric f, Epsilon a) => f a -> f a
normalize v = if nearZero l || nearZero (1-l) then v else fmap (/sqrt l) v
where l = quadrance v
What this means is that if the magnitude of your points is really close to 0 you're going to end up with the wrong value. To avoid this you can write your own normalize function without this check as
normalize' :: (Floating a, Metric f) => f a -> f a
normalize' v = fmap (/ sqrt l) v where l = quadrance v
And with any luck it should solve your problem.
Another way around this might be to scale up your values, perform the computations, then scale them back down, something like
normalize' factor = (* factor) . normalize . (/ factor)
So you might call
normalize' 10e-10 (V3 1e-10 2e-10 3e-10)
instead, but this could easily introduce rounding errors due to how IEEE floating point numbers are stored.
EDIT: As cchalmers points out this is already implemented as signorm in Linear.Metric, so use that function instead.
Related
I came across a nice post on SO by #amalloy while looking for hylomorhism examples, that illustrate recursion scheme (RS) usage with useful discussion and full implementation:
{-# LANGUAGE DeriveFunctor #-}
import Control.Arrow ( (>>>), (<<<) )
newtype Term f = In {out :: f (Term f)}
type Algebra f a = f a -> a
type Coalgebra f a = a -> f a
cata :: (Functor f) => Algebra f a -> Term f -> a
cata fn = out >>> fmap (cata fn) >>> fn
ana :: (Functor f) => Coalgebra f a -> a -> Term f
ana f = In <<< fmap (ana f) <<< f
hylo :: Functor f => Algebra f b -> Coalgebra f a -> a -> b
hylo alg coalg = ana coalg >>> cata alg
data ChangePuzzle a = Solved Cent
| Pending {spend, forget :: a}
deriving Functor
type Cent = Int
type ChangePuzzleArgs = ([Cent], Cent)
coins :: [Cent]
coins = [50, 25, 10, 5, 1]
divide :: Coalgebra ChangePuzzle ChangePuzzleArgs
divide (_, 0) = Solved 1
divide ([], _) = Solved 0
divide (coins#(x:xs), n) | n < 0 = Solved 0
| otherwise = Pending (coins, n - x) (xs, n)
conquer :: Algebra ChangePuzzle Cent
conquer (Solved n) = n
conquer (Pending a b) = a + b
waysToMakeChange :: ChangePuzzleArgs -> Int
waysToMakeChange = hylo conquer divide
The code works as expected. Despite having some vague intuition for the RS aspect already, I am still wondering:
since this is about counting combinations, why Solved Cent and not Solved Int? (This may sound like a nitpic, if it is even a reasonable question, but I am hoping it may be the root of the rest of the uncertainty, below, although I suspect I missed something more fundamental!).
since we're later summing, in divide, Solved 0/1 presumably signifies failure/success?
in conquer, what does it mean to add, a and b, of Pending? What do those 2 values (as Cents) signify, and what would their sum mean in this context?
in conquer, I would have expected we just need to sum the Solveds, and the author touches on this, but it's not clear, yet, how the Pending case is contributing (eg fixing conquer (Pending a b) = 11 does have an adverse impact on functionality, and it is probably a clue that waysToMakeChange returns 11, or whatever constant that case is fixed to).
in conquer, a and b are Cents, whereas in divide they're ChangePuzzleArgs (aka ([Cent], Cent)) - where does that transformation occur?
Note: being new to SO, I was not able to comment below the original answer, which may have been more appropriate, but I hope this is also useful as is.
since this is about counting combinations, why Solved Cent and not Solved Int? (This may sound like a nitpic, if it is even a reasonable question, but I am hoping it may be the root of the rest of the uncertainty, below, although I suspect I missed something more fundamental!).
I would also use Int here.
since we're later summing, in divide, Solved 0/1 presumably signifies failure/success?
Yes, but it's slightly more than that. Solved 0 means "there are exactly 0 ways to generate that change amount" (i.e., failure), while Solved 1 means "there is exactly 1 way to generate that change amount" (i.e., success). In the latter case, not only we mean "success", but we also report that there is only one way to solve the task.
in conquer, what does it mean to add, a and b, of Pending? What do those 2 values (as Cents) signify, and what would their sum mean in this context?
Essentially, Pending a b with a,b::Int means "the number of ways to generate that change amount can be split into two disjoint sets, the first one having a elements, and the second one having b elements".
When we divide, we return Pending ... ... to split the problem into two disjoint subcases, (coins, n - x) and (xs, n). Here coins=(x:xs). We split according to whether we want to use coin x at least one time (hence we need to generate n-x with all the coins), or we don't want to use it at all (hence we need to generate n with the other coins, only).
in conquer, I would have expected we just need to sum the Solveds, and the author touches on this, but it's not clear, yet, how the Pending case is contributing (eg fixing conquer (Pending a b) = 11 does have an adverse impact on functionality, and it is probably a clue that waysToMakeChange returns 11, or whatever constant that case is fixed to).
Summing all the Solved ... is what we do. The cata magic essentially replaces the straightforward recursive sum
foo (Solved n) = n
foo (Pending case1 case2) = foo case1 + foo case2
with cata conquer where
conquer (Solved n) = n
conquer (Pending a b) = a + b
The magic of cata makes it so that inside Pending, we do not find subtrees upon which we want to recurse, but the result of the recursion, already computed.
in conquer, a and b are Cents, whereas in divide they're ChangePuzzleArgs (aka ([Cent], Cent)) - where does that transformation occur?
This can be indeed subtle at first. I'll provide only a rough intuition.
After ana divide we produce a result in a fixed point of the functor ChangePuzzle. Note how ana at the end returns Term ChangePuzzle, which is the fixed point. There, the pair ([Cent], Cent) magically disappears.
Dually, the Int reappears when we use cata, even if we started from Term ChangePuzzle.
Very roughly, you can think of Term ChangePuzzle as the infinite nesting
ChangePuzzle (ChangePuzzle (ChangePuzzle ( ....
which is coherent with the fact that such a tree might be arbitrarily nested. There, the "argument" of ChangePuzzle essentially disappears.
How do we get the final Int then? Well, we get that since Solved always takes an Int argument, and not an a argument. This provides that base case that makes the final cata recursion work.
A while back a friend wanted help with a program that could solve for the roots of functions using Newton's method, and naturally for that I needed some way to calculate the derivative of a function numerically, and this is what I came up with:
deriv f x = (f (x+h) - f x) / h where h = 0.00001
Newton's method was a fairly easy thing to implement, and it works rather well. But now I've started to wonder - Is there some way I could use this function to solve partial derivatives in a numerical manner, or is that something that would require a full-on CAS? I would post my attempts but I have absolutely no clue what to do yet.
Please keep in mind that I am new to Haskell. Thank you!
You can certainly do much the same thing as you already implemented, only with multivariate perturbation instead. But first, as you should always do with top-level functions, add a type signature:
deriv :: (Double -> Double) -> Double -> Double
That's not the most general signature possible, but probably sufficiently general for everything you'll need. I'll call
type ℝ = Double
in the following for brevity, i.e.
deriv :: (ℝ -> ℝ) -> ℝ -> ℝ
Now what you want is, for example in ℝ²
grad :: ((ℝ,ℝ) -> ℝ) -> (ℝ,ℝ) -> (ℝ,ℝ)
grad f (x,y) = ((f (x+h,y) - f (x,y)) / h, (f (x,y+h) - f (x,y)) / h)
where h = 0.00001
It's awkward to have to write out the components individually and make the definition specific to a particular-dimensional vector space. A generic way of doing it:
import Data.VectorSpace
import Data.Basis
grad :: (HasBasis v, Scalar v ~ ℝ) => (v -> ℝ) -> v -> v
grad f x = recompose [ (e, (f (x ^+^ h*^basisValue b) - f x) ^/ h)
| (e,_) <- decompose x ]
where h = 0.00001
Note that this pre-chosen-step–finite-differentiation is always a tradeoff between inaccuracy from higher-order terms and from floating-point errors, so definitely check out automatic differentiation.
This is called automatic differentiation and there is a lot of really neat work in this area in Haskell, though I don't know how accessible it is.
From the wiki page:
A paper Beautiful Differentiation and the corresponding talk.
Forward mode libraries: ad, fad, vector-space, Data.Ring.Module.AutomaticDifferentiation
Reverse mode libraries: also ad, rad
I need a second order function pairApply that applies a binary function f to all unique pairs of a list-like structure and then combines them somehow. An example / sketch:
pairApply (+) f [a, b, c] = f a b + f a c + f b c
Some research leads me to believe that Data.Vector.Unboxed probably will have good performance (I will also need fast access to specific elements); also it necessary for Statistics.Sample, which would come in handy further down the line.
With this in mind I have the following, which almost compiles:
import qualified Data.Vector.Unboxed as U
pairElement :: (U.Unbox a, U.Unbox b)
=> (U.Vector a)
-> (a -> a -> b)
-> Int
-> a
-> (U.Vector b)
pairElement v f idx el =
U.map (f el) $ U.drop (idx + 1) v
pairUp :: (U.Unbox a, U.Unbox b)
=> (a -> a -> b)
-> (U.Vector a)
-> (U.Vector (U.Vector b))
pairUp f v = U.imap (pairElement v f) v
pairApply :: (U.Unbox a, U.Unbox b)
=> (b -> b -> b)
-> b
-> (a -> a -> b)
-> (U.Vector a)
-> b
pairApply combine neutral f v =
folder $ U.map folder (pairUp f v) where
folder = U.foldl combine neutral
The reason this doesn't compile is that there is no Unboxed instance of a U.Vector (U.Vector a)). I have been able to create new unboxed instances in other cases using Data.Vector.Unboxed.Deriving, but I'm not sure it would be so easy in this case (transform it to a tuple pair where the first element is all the inner vectors concatenated and the second is the length of the vectors, to know how to unpack?)
My question can be stated in two parts:
Does the above implementation make sense at all or is there some quick library function magic etc that could do it much easier?
If so, is there a better way to make an unboxed vector of vectors than the one sketched above?
Note that I'm aware that foldl is probably not the best choice; once I've got the implementation sorted I plan to benchmark with a few different folds.
There is no way to define a classical instance for Unbox (U.Vector b), because that would require preallocating a memory area in which each element (i.e. each subvector!) has the same fixed amount of space. But in general, each of them may be arbitrarily big, so that's not feasible at all.
It might in principle be possible to define that instance by storing only a flattened form of the nested vector plus an extra array of indices (where each subvector starts). I once briefly gave this a try; it actually seems somewhat promising as far as immutable vectors are concerned, but a G.Vector instance also requires a mutable implementation, and that's hopeless for such an approach (because any mutation that changes the number of elements in one subvector would require shifting everything behind it).
Usually, it's just not worth it, because if the individual element vectors aren't very small the overhead of boxing them won't matter, i.e. often it makes sense to use B.Vector (U.Vector b).
For your application however, I would not do that at all – there's no need to ever wrap the upper element-choices in a single triangular array. (And it would be really bad for performance to do that, because it make the algorithm take O (n²) memory rather than O (n) which is all that's needed.)
I would just do the following:
pairApply combine neutral f v
= U.ifoldl' (\acc i p -> U.foldl' (\acc' q -> combine acc' $ f p q)
acc
(U.drop (i+1) v) )
neutral v
This corresponds pretty much to the obvious nested-loops imperative implementation
pairApply(combine, b, f, v):
for(i in 0..length(v)-1):
for(j in i+1..length(v)-1):
b = combine(b, f(v[i], v[j]);
return b;
My answer is basically the same as leftaroundabout's nested-loops imperative implementation:
pairApply :: (Int -> Int -> Int) -> Vector Int -> Int
pairApply f v = foldl' (+) 0 [f (v ! i) (v ! j) | i <- [0..(n-1)], j <- [(i+1)..(n-1)]]
where n = length v
As far as I know, I do not see any performance issue with this implementation.
Non-polymorphic for simplicity.
Are there any libraries for sequential non-linear optimization with upper and lower bounds, as well as inequality constraints, that are written in or easily callable from Haskell?
The bindings-levmar package provides bindings to a C Levenberg-Marquardt optimizer.
A quick grep of Hackage suggests that nonlinear-optimization is the best (only) already-written thing; however, it doesn't seem to include anything for bounded optimization.
Your best bet seems to be one of these (in order of increasing attractiveness):
Start your own project.
Extend the above package.
Find a decent C library and learn enough FFI to bind to it.
I know the OP asked for a general optimization library, where my experience is:
The levmar package depends on the blas and lapack libraries and that makes things very complicated to install, I didn't manage to install it such that ghci is still working on Windows.
The nonlinear-optimization package requires a gradient function
The optimization package seems also to require gradients, though I couldn't figure out how to actually use it.
Besides, all of the mentioned packages do not seem to have any real documentation.
Fortunately, for simple problems a simple solution can be enough. If you want to optimize a one-dimensional, smooth and convex function which has a single bracketed extremum but you do not know a gradient function (see below if you do1) then a simple method like Golden Section Search will do.
Translated from the Wikipedia page:
import Data.Maybe (fromMaybe)
-- 1 / phi
invphi = (sqrt 5 - 1) / 2
-- 1 / phi^2
invphi2 = (3 - sqrt 5) / 2
-- | Enable optional arguments syntax. Use with Maybe a as parameter type, then in the function write param // defaultValue
(//) :: Maybe a -> a -> a
(//) = flip fromMaybe
-- Just a wrapper function because of all the ugly Nothing's of the recursive function
goldenSectionSearch f a b tolerance = goldenSectionSearchRecursive f a b tolerance Nothing Nothing Nothing Nothing Nothing
-- | Golden section search, recursive.
-- Given a function f with a single local maximum in the interval [a, b], golden section search returns a subset interval [c, d] that contains the maximum with d-c <= tolerance
-- Taken from the python implementation at https://en.wikipedia.org/wiki/Golden-section_search
goldenSectionSearchRecursive ::
(Double -> Double) -- ^ Function with a single maximum in [a, b]
-> Double -- ^ One side of the interval
-> Double -- ^ Other side of the interval
-> Double -- ^ Tolerance
-> Maybe Double -- ^ h, Current search interval
-> Maybe Double -- ^ c, New left interval point. If Nothing, a new point is chosen.
-> Maybe Double -- ^ d, New right interval point.
-> Maybe Double -- ^ f(c), Function value at c
-> Maybe Double -- ^ f(d), Function value at d
-> (Double, Double) -- ^ The interval in which the maximum is
goldenSectionSearchRecursive f a' b' tolerance h' c' d' fc' fd'
| h < tolerance = (a, b)
| fc > fd = goldenSectionSearchRecursive f a d tolerance (Just (h * invphi)) Nothing (Just c) Nothing (Just fc)
| otherwise = goldenSectionSearchRecursive f c b tolerance (Just (h * invphi)) (Just d) Nothing (Just fd) Nothing
where
a = min a' b'
b = max a' b'
h = h' // (b - a)
c = c' // (a + invphi2 * h)
d = d' // (a + invphi * h)
fc = fc' // f c
fd = fd' // f d
and then you call with goldenSectionSearch (\x -> -(x-2)^2) 1 5 1e-5 which returns (1.9999959837979107,2.0000050911830893). This simple function of course would be much easier to solve by hand, but it's just an example.
PS Interesting about Golden Section Search is that the convergence rate is exactly known: on each iteration the length of the interval in which the optimum resides is divided by the golden ratio.
PPS I put it on GitHub
[1] Note that if you know a gradient function, equating it to zero and applying a root finding method is often much faster. For example in one dimension, Will Ness pointed to his answer which has a simple method with a faster convergence rate than the Golden Section Search.
You could also use one of the mentioned packages which require a gradient function, of course.
I'm giving Haskell a go at the moment and struggling to rewrite loops in terms of recursions.
I am trying to write a basic integrator which takes some function f(x) and integrates it over the range [a, b] via the Midpoint Method. The integrating function takes three parameters N, a, and b, where N is the number of rectangles being used to approximate the integral.
Wolfram Alpha link
When I try to compile this with GHCI I get a lot of abstract error messages and I don't really know where to start. Lots of 'Out of Scopes' and several 'Multiple Declarations of Main.a [or b]'.
Thanks
MPInt 1 a b = DELTA 1 -- Base case
MPInt N a b = (MPInt (N-1) a b) + DELTA
where
dX = (b - a) / N
DELTA = dX * f (a + dX * (N+0.5))
f :: (Num a) => a -> a
f x = x^2
You've named your function MPInt. Only the names of modules, classes, types, and constructors can begin with capital letters; values (including functions) must begin with lowercase letters (or an underscore, or certain punctuation marks). The same problem also applies to N and DELTA.
You use DELTA in the definition of the first case of MPInt, yet it's defined as part of the second case. A where clause only applies to the expression immediately before it (in this case, the MPInt N a b = ... definition). (Also, this first use of DELTA treats it as a function, yet the second use and its definition have it as a numeric value).