Are there any libraries for sequential non-linear optimization with upper and lower bounds, as well as inequality constraints, that are written in or easily callable from Haskell?
The bindings-levmar package provides bindings to a C Levenberg-Marquardt optimizer.
A quick grep of Hackage suggests that nonlinear-optimization is the best (only) already-written thing; however, it doesn't seem to include anything for bounded optimization.
Your best bet seems to be one of these (in order of increasing attractiveness):
Start your own project.
Extend the above package.
Find a decent C library and learn enough FFI to bind to it.
I know the OP asked for a general optimization library, where my experience is:
The levmar package depends on the blas and lapack libraries and that makes things very complicated to install, I didn't manage to install it such that ghci is still working on Windows.
The nonlinear-optimization package requires a gradient function
The optimization package seems also to require gradients, though I couldn't figure out how to actually use it.
Besides, all of the mentioned packages do not seem to have any real documentation.
Fortunately, for simple problems a simple solution can be enough. If you want to optimize a one-dimensional, smooth and convex function which has a single bracketed extremum but you do not know a gradient function (see below if you do1) then a simple method like Golden Section Search will do.
Translated from the Wikipedia page:
import Data.Maybe (fromMaybe)
-- 1 / phi
invphi = (sqrt 5 - 1) / 2
-- 1 / phi^2
invphi2 = (3 - sqrt 5) / 2
-- | Enable optional arguments syntax. Use with Maybe a as parameter type, then in the function write param // defaultValue
(//) :: Maybe a -> a -> a
(//) = flip fromMaybe
-- Just a wrapper function because of all the ugly Nothing's of the recursive function
goldenSectionSearch f a b tolerance = goldenSectionSearchRecursive f a b tolerance Nothing Nothing Nothing Nothing Nothing
-- | Golden section search, recursive.
-- Given a function f with a single local maximum in the interval [a, b], golden section search returns a subset interval [c, d] that contains the maximum with d-c <= tolerance
-- Taken from the python implementation at https://en.wikipedia.org/wiki/Golden-section_search
goldenSectionSearchRecursive ::
(Double -> Double) -- ^ Function with a single maximum in [a, b]
-> Double -- ^ One side of the interval
-> Double -- ^ Other side of the interval
-> Double -- ^ Tolerance
-> Maybe Double -- ^ h, Current search interval
-> Maybe Double -- ^ c, New left interval point. If Nothing, a new point is chosen.
-> Maybe Double -- ^ d, New right interval point.
-> Maybe Double -- ^ f(c), Function value at c
-> Maybe Double -- ^ f(d), Function value at d
-> (Double, Double) -- ^ The interval in which the maximum is
goldenSectionSearchRecursive f a' b' tolerance h' c' d' fc' fd'
| h < tolerance = (a, b)
| fc > fd = goldenSectionSearchRecursive f a d tolerance (Just (h * invphi)) Nothing (Just c) Nothing (Just fc)
| otherwise = goldenSectionSearchRecursive f c b tolerance (Just (h * invphi)) (Just d) Nothing (Just fd) Nothing
where
a = min a' b'
b = max a' b'
h = h' // (b - a)
c = c' // (a + invphi2 * h)
d = d' // (a + invphi * h)
fc = fc' // f c
fd = fd' // f d
and then you call with goldenSectionSearch (\x -> -(x-2)^2) 1 5 1e-5 which returns (1.9999959837979107,2.0000050911830893). This simple function of course would be much easier to solve by hand, but it's just an example.
PS Interesting about Golden Section Search is that the convergence rate is exactly known: on each iteration the length of the interval in which the optimum resides is divided by the golden ratio.
PPS I put it on GitHub
[1] Note that if you know a gradient function, equating it to zero and applying a root finding method is often much faster. For example in one dimension, Will Ness pointed to his answer which has a simple method with a faster convergence rate than the Golden Section Search.
You could also use one of the mentioned packages which require a gradient function, of course.
Related
I am trying to implement a generator, selector pattern to approximately calculate square roots in haskell
My generator looks like this:
generator :: (Double -> Double) -> Double -> [Double]
generator f a = generator f (f a)
My selector:
selector :: Double -> [Double] -> Double
selector eps (a : b : r)
| abs(a - b) <= eps = b
| otherwise = selector eps (b : r)
And the approx function:
next :: Double -> Double -> Double
next n x = (x + n/x) / 2
Calling this like selector 0.1 (generator (next 5) 2)
should give me ...(next 5( next 5 (next 5 2))) so [2.25, 2.23611111111111, 2.2360679779158,...] since my eps parameter is 0.1 abs(a - b) <= eps should be true on the first execution giving me 2.23611111111111 as a result. I do however end in a endless loop.
Could somebody explain to me what is wrong in the implementation of those functions?
Thanks in advance
This definition
generator f a = generator f (f a)
never generates any list elements: it gets stuck into an infinite recursion instead. You probably want
generator f a = a : generator f (f a)
which makes a to be the first element, followed by all the others we generate using recursion.
It could also be beneficial to avoid putting unevaluated thunks in the list. To avoid that, one could use
generator f a = a `seq` (a : generator f (f a))
so that a is evaluated early. This should not matter much in your code, since the
selector immediately evaluates the thunks as soon as they are generated.
Your generator function is missing the a:, as chi's answer correctly points out. However, there's a better solution than just adding that. Get rid of generator altogether, and use the built-in method iterate instead (or iterate' from Data.List if you want to avoid unevaluated thunks). These methods have the same behavior that you want from generate, but support optimizations like list fusion that your own method won't. And of course, there's also the advantage that it's one less function that you have to write and maintain.
A while back a friend wanted help with a program that could solve for the roots of functions using Newton's method, and naturally for that I needed some way to calculate the derivative of a function numerically, and this is what I came up with:
deriv f x = (f (x+h) - f x) / h where h = 0.00001
Newton's method was a fairly easy thing to implement, and it works rather well. But now I've started to wonder - Is there some way I could use this function to solve partial derivatives in a numerical manner, or is that something that would require a full-on CAS? I would post my attempts but I have absolutely no clue what to do yet.
Please keep in mind that I am new to Haskell. Thank you!
You can certainly do much the same thing as you already implemented, only with multivariate perturbation instead. But first, as you should always do with top-level functions, add a type signature:
deriv :: (Double -> Double) -> Double -> Double
That's not the most general signature possible, but probably sufficiently general for everything you'll need. I'll call
type ℝ = Double
in the following for brevity, i.e.
deriv :: (ℝ -> ℝ) -> ℝ -> ℝ
Now what you want is, for example in ℝ²
grad :: ((ℝ,ℝ) -> ℝ) -> (ℝ,ℝ) -> (ℝ,ℝ)
grad f (x,y) = ((f (x+h,y) - f (x,y)) / h, (f (x,y+h) - f (x,y)) / h)
where h = 0.00001
It's awkward to have to write out the components individually and make the definition specific to a particular-dimensional vector space. A generic way of doing it:
import Data.VectorSpace
import Data.Basis
grad :: (HasBasis v, Scalar v ~ ℝ) => (v -> ℝ) -> v -> v
grad f x = recompose [ (e, (f (x ^+^ h*^basisValue b) - f x) ^/ h)
| (e,_) <- decompose x ]
where h = 0.00001
Note that this pre-chosen-step–finite-differentiation is always a tradeoff between inaccuracy from higher-order terms and from floating-point errors, so definitely check out automatic differentiation.
This is called automatic differentiation and there is a lot of really neat work in this area in Haskell, though I don't know how accessible it is.
From the wiki page:
A paper Beautiful Differentiation and the corresponding talk.
Forward mode libraries: ad, fad, vector-space, Data.Ring.Module.AutomaticDifferentiation
Reverse mode libraries: also ad, rad
I've spent a few minutes debugging a problem that tracked down to "Linear" truncating values that are close to zero when using "Linear.normalize". Specifically, I was taking the cross product of very small triangles and normalizing the result, which, surprisingly, behaved wrongly until I noticed what was wrong and multiplied the cross product by 10000.
Why is that even necessary? How can I get rid of that behavior?
Edit: just for fun, here is a video of the bug. Notice that the sphere loses the color when the number of triangles approximating it is big enough? Yes, good luck debugging that...!
Looking at the source for normalize, you'll see that it's defined as
-- | Normalize a 'Metric' functor to have unit 'norm'. This function
-- does not change the functor if its 'norm' is 0 or 1.
normalize :: (Floating a, Metric f, Epsilon a) => f a -> f a
normalize v = if nearZero l || nearZero (1-l) then v else fmap (/sqrt l) v
where l = quadrance v
What this means is that if the magnitude of your points is really close to 0 you're going to end up with the wrong value. To avoid this you can write your own normalize function without this check as
normalize' :: (Floating a, Metric f) => f a -> f a
normalize' v = fmap (/ sqrt l) v where l = quadrance v
And with any luck it should solve your problem.
Another way around this might be to scale up your values, perform the computations, then scale them back down, something like
normalize' factor = (* factor) . normalize . (/ factor)
So you might call
normalize' 10e-10 (V3 1e-10 2e-10 3e-10)
instead, but this could easily introduce rounding errors due to how IEEE floating point numbers are stored.
EDIT: As cchalmers points out this is already implemented as signorm in Linear.Metric, so use that function instead.
How to numerically integrate complex, complex-valued functions in Haskell?
Are there any existing libraries for it? numeric-tools operates only on reals.
I am aware that on complex plane there's only line integrals, so the interface I am interested in is something like this:
i = integrate f x a b precision
to calculate integral along straight line from a to b of function f on point x.
i, x, a, b are all of Complex Double or better Num a => Complex a type.
Please... :)
You can make something like this yourself. Suppose you have a function realIntegrate of type (Double -> Double) -> (Double,Double) -> Double, taking a function and a tuple containing the lower and upper bounds, returning the result to some fixed precision. You could define realIntegrate f (lo,hi) = quadRomberg defQuad (lo,hi) f using numeric-tools, for example.
Then we can make your desired function as follows - I'm ignoring the precision for now (and I don't understand what your x parameter is for!):
integrate :: (Complex Double -> Complex Double) -> Complex Double -> Complex Double -> Complex Double
integrate f a b = r :+ i where
r = realIntegrate realF (0,1)
i = realIntegrate imagF (0,1)
realF t = realPart (f (interpolate t)) -- or realF = realPart . f . interpolate
imagF t = imagPart (f (interpolate t))
interpolate t = a + (t :+ 0) * (b - a)
So we express the path from a to b as a function on the real interval from 0 to 1 by linear interpolation, take the value of f along that path, integrate the real and imaginary parts separately (I don't know if this can give numerically badly behaving results, though) and reassemble them into the final answer.
I haven't tested this code as I don't have numeric-tools installed, but at least it typechecks :-)
I'm giving Haskell a go at the moment and struggling to rewrite loops in terms of recursions.
I am trying to write a basic integrator which takes some function f(x) and integrates it over the range [a, b] via the Midpoint Method. The integrating function takes three parameters N, a, and b, where N is the number of rectangles being used to approximate the integral.
Wolfram Alpha link
When I try to compile this with GHCI I get a lot of abstract error messages and I don't really know where to start. Lots of 'Out of Scopes' and several 'Multiple Declarations of Main.a [or b]'.
Thanks
MPInt 1 a b = DELTA 1 -- Base case
MPInt N a b = (MPInt (N-1) a b) + DELTA
where
dX = (b - a) / N
DELTA = dX * f (a + dX * (N+0.5))
f :: (Num a) => a -> a
f x = x^2
You've named your function MPInt. Only the names of modules, classes, types, and constructors can begin with capital letters; values (including functions) must begin with lowercase letters (or an underscore, or certain punctuation marks). The same problem also applies to N and DELTA.
You use DELTA in the definition of the first case of MPInt, yet it's defined as part of the second case. A where clause only applies to the expression immediately before it (in this case, the MPInt N a b = ... definition). (Also, this first use of DELTA treats it as a function, yet the second use and its definition have it as a numeric value).