It is often desirable to obtain the solution to a mathematical problem in closed form, that is, as an expression that contains generally-accepted functions like polynomials, rational and irrational functions, roots, and exponentials and logarithms. One justification I often hear is that, when known functions are involved, it easier to visualize the behavior of the function. Another justification is that it is less computationally demanding to evaluate the function at a set of points. While I certainly agree with the first justification, is the second justification reasonable? For example:
Does it take a longer time to compute a modified Bessel function of the first kind and fifth order for 10 points than to compute an exponential?
Does it take a longer time to compute an exponential integral than to compute an exponential?
My intuition is that in all three cases, a Taylor series expansion around the desired point is formed, so it comes down to evaluating a polynomial, some other polynomial, or its antiderivative.
One justification I often hear is that, when known functions are involved, it easier to visualize the behavior of the function.
Another justification is that it is less computationally demanding to evaluate the function at a set of points.
That's only if the function is "simple", not closed. You can construct closed forms which are arbitrarily complex and demanding computationally.
I can imagine two genuine advantages of closed form solutions:
Since most programming languages have support for sqrt, sin and the like, closed form solutions are easily representable in code.
If a solution has closed form then there will be a number of algebraic steps that you can follow to get to the solution – an algebraic (or perhaps trigonometric) solution algorithm. Those algorithms can also only contain "closed form" steps, so they'll be fairly easily implemented.
If you know that the solution of your problem probably doesn't have closed form, you'll have to reside to an entirely different approach for solving it. This can get quite tricky: The Babylonians were able to solve quadratic equations 2000 BC, and it took more than another 3000 years until roots of polynomials of arbitrary order could be solved – with numerics, not algebra.
Related
Suppose we're given some sort of graph where the feasible region of our optimization problem is given. For example: here is an image
How would I go on about constructing these constraints in an integer optimization problem? Anyone got any tips? Thanks!
Mate, I agree with the others that you should be a little more specific than that paint-ish picture ;). In particular you are neither specifying any objective/objective direction nor are you giving any context, what about this graph should be integer-variable related, except for the existence of disjunctive feasible sets, which may be modeled by MIP-techniques. It seems like your problem is formalization of what you conceptualized. However, in case you are just being lazy and are just interested in modelling disjunctive regions, you should be looking into disjunctive programming techniques, such as "big-M" (Note: big-M reformulations can be problematic). You should be aiming at some convex-hull reformulation if you can attain one (fairly easily).
Back to your picture, it is quite clear that you have a problem in two real dimensions (let's say in R^2), where the constraints bounding the feasible set are linear (the lines making up the feasible polygons).
So you know that you have two dimensions and need two real continuous variables, say x[1] and x[2], to formulate each of your linear constraints (a[i,1]*x[1]+a[i,2]<=rhs[i] for some index i corresponding to the number of lines in your graph). Additionally your variables seem to be constrained to the first orthant so x[1]>=0 and x[2]>=0 should hold. Now, to add disjunctions you want some constraints that only hold when a certain condition is true. Therefore, you can add two binary decision variables, say y[1],y[2] and an additional constraint y[1]+y[2]=1, to tell that only one set of constraints can be active at the same time. You should be able to implement this with the help of big-M by reformulating the constraints as follows:
If you bound things from above with your line:
a[i,1]*x[1]+a[i,2]-rhs[i]<=M*(1-y[1]) if i corresponds to the one polygon,
a[i,1]*x[1]+a[i,2]-rhs[i]<=M*(1-y[2]) if i corresponds to the other polygon,
and if your line bounds things from below:
-M*(1-y[1])<=-a[i,1]*x[1]-a[i,2]+rhs[i] if i corresponds to the one polygon,
-M*(1-y[1])<=-a[i,1]*x[1]-a[i,2]+rhs[i] if i corresponds to the other polygon.
It is important that M is sufficiently large, but not too large to cause numerical issues.
That being said, I am by no means an expert on these disjunctive programming techniques, so feel free to chime in, add corrections or make things clearer.
Also, a more elaborate question typically yields more elaborate and satisfying answers ;) If you had gone to the effort of making up a true small example problem you likely would have gotten a full formulation of your problem or even an executable piece of code in no time.
I wrote an application in Haskell that calls Z3 solver to solve constrains with some complex formulas. Thanks to Haskell I can quickly switch the data type I'm working with.
When using SBV's AlgReal type for computations, I get results in sensible time, however switching to Float or Double types makes Z3 consume ~2Gb of RAM and doesn't result even in 30 minutes.
Is this expected that producing floating point solutions require much more time, or it is some mistake on my side?
As with any question regarding solver performance, it is impossible to make generalizations. Christoph Wintersteiger (https://stackoverflow.com/users/869966/christoph-wintersteiger) would be the expert on this to opine, but I'm not sure how closely he follows this group. Chris: If you're reading this, I'd love to hear your thoughts!
There's also the risk of comparing apples-to-oranges here: Reals and floats are two completely different logics, with different decision procedures, heuristics, algorithms, etc. I'm sure you can find problems where one outperforms the other, with no clear "performance" winner.
Having said all that, here are some things that make floating-point (FP) tricky:
Rewriting is almost impossible with FP, since rules like associativity simply
don't hold for FP addition/multiplication. So, there are fewer opportunities for
simplification before bit-blasting.
Similarly a * 1/a == 1 doesn't hold for floats. Neither does x + 1 /= x or (x + a == x) -> (a == 0) and many other "obvious" simplifications that you'd love to be able to make. All of this complicates reasoning.
Existence of NaN values make equality non-reflexive: Nothing compares equal to NaN including itself. So, substitution of equals-for-equals is also problematic and requires side conditions.
Likewise, the existence of +0 and -0, which compare equal but behave differently due to rounding complicate matters. The typical example is x == 0 -> fma(a, b, x) == a * b doesn't hold (where fma is fused multiply-add), because depending on the sign of zero these two expressions can produce different values for different rounding modes.
Combination of floats with integers and reals introduce non-linearity, which is always a soft-spot for solvers. So, if you're using FP, it is advisable not to mix it with other theories as the combination itself creates extra complexity.
Operations like multiplication, division, and remainder (yes, there's a floating-point remainder operation!) are inherently very complex and lead to extremely large SAT formulas. In particular, multiplication is a known operation that challenges most SAT/BDD engines, due to lack of any good variable ordering and splitting heuristics. Solvers end-up bit-blasting fairly quickly, resulting in extremely large state-spaces. I have observed that solvers have a hard time dealing with FP division and remainder operations even when the input is completely concrete, imagine what happens when they are fully symbolic!
The logic of reals have a decision procedure with a double-exponential complexity. However, techniques like Fourier-Motzkin elimination (https://en.wikipedia.org/wiki/Fourier%E2%80%93Motzkin_elimination), while remaining exponential, perform really well in practice.
FP solvers are relatively new, and it's a niche area with nascent research. So existing solvers tend to be quite conservative and quickly bit-blast and reduce the problem to bit-vector logic. I would expect them to improve over time, just like all the other theories did.
Again, I want to emphasize comparing solver performance on these two different logics is misguided as they are totally different beasts. But hopefully, the above points illustrate why floating-point is tricky in practice.
A great paper to read about the treatment of IEEE754 floats in SMT solvers is: http://smtlib.cs.uiowa.edu/papers/BTRW14.pdf. You can see the myriad of operations it supports and get a sense of the complexity.
CLP(FD) allows user to set the domain for every wannabe-integer variable, so it's able to solve equations.
So far so good.
However you can't do the same in CLP(R) or similar languages (where you can do only simple inferences). And it's not hard to understand why: the fractional part of a number may have an almost infinite region, putted down by an implementation limit. This mean the search space will be too large to make any practical use for a solver which deals with floats like with integers. So it's the user task to write generator in CLP(R) and set constraint guards where needed to get variables instantiated with numbers (if simple inference is not possible).
So my question here: is there any CLP(FD)-like language over reals? I think it could be implemented by use of number rounding, searching and following incremental approximation.
There are at least some major CLP(FD) solvers that support real (decision) variables:
Gecode
JaCoP
ECLiPSe CLP (ic library)
Choco (using Ibex)
(The first three also support var float in MiniZinc.)
The answser to your question is yes. There is Constraint-based Solvers dedicated for floating numbers. I do not have a list of solvers but I know that that ibex http://www.ibex-lib.org is a library allowing the use of floats. You should also have a look at SMT-Solvers implementing the Real-Theory (http://smtlib.cs.uiowa.edu/solvers.shtml).
I'm looking for a functional data structure that represents finite bijections between two types, that is space-efficient and time-efficient.
For instance, I'd be happy if, considering a bijection f of size n:
extending f with a new pair of elements has complexity O(ln n)
querying f(x) or f^-1(x) has complexity O(ln n)
the internal representation for f is more space efficient than having 2 finite maps (representing f and its inverse)
I am aware of efficient representation of permutations, like this paper, but it does not seem to solve my problem.
Please have a look at my answer for a relatively similar question. The provided code can handle general NxM relations, but also be specialized to just bijections (just as you would for a binary search tree).
Pasting the answer here for completeness:
The simplest way is to use a pair of unidirectional maps. It has some cost, but you won't get much better (you could get a bit better using dedicated binary trees, but you have a huge complexity cost to pay if you have to implement it yourself). In essence, lookups will be just as fast, but addition and deletion will be twice as slow. Which isn't so bad for a logarithmic operation. Another advantage of this technique is that you can use specialized maps types for the key or value type if you have one available. You won't get as much flexibility with a specific generalist data structure.
A different solution is to use a quadtree (instead of considering a NxN relation as a pair of 1xN and Nx1 relations, you see it as a set of elements in the cartesian product (Key*Value) of your types, that is, a spatial plane), but it's not clear to me that the time and memory costs are better than with two maps. I suppose it needs to be tested.
Although it doesn't satisfy your third requirement, bimaps seem like the way to go. (They just make two finite maps, one in each direction, convenient to use.)
Most programming languages give 2 as the answer to square root of 4. However, there are two answers: 2 and -2. Is there any particular reason, historical or otherwise, why only one answer is usually given?
Because:
In mathematics, √x commonly, unless otherwise specified, refers to the principal (i.e. positive) root of x [http://mathworld.wolfram.com/SquareRoot.html].
Some languages don't have the ability to return more than one value.
Since you can just apply negation, returning both would be redundant.
If the square root method returned two values, then one of those two would practically always be discarded. In addition to wasting memory and complexity on the extra return value, it would be little used. Everyone knows that you can multiple the answer returned by -1 and get the other root.
I expect that only mathematical languages would return multiple values here, perhaps as an array or matrix. But for most general-purpose programming languages, there is negligible gain and non-negligible cost to doing as you suggest.
Some thoughts:
Historically, functions were defined as procedures which returned a single value.
It would have been fiddly (using primitive programming constructs) to define a clean function which returned multiple values like this.
There are always exceptions to the rule:
0 for example only has a single root (0).
You cannot take the square root of a negative number (unless the language supports complex numbers). This could be treated as an exception (like "divide by 0") in languages which don't support imaginary numbers or the complex number system.
It is usually simple to deduce the 2 square roots (simply negate the value returned by the function). This was probably left as an exercise by the caller of the sqrt() function, if their domain depended on dealing with both the positive (+) and negative (-) roots.
It's easier to return one number than to return two. Most engineering decisions are made in this manner.
There are many functions which only return 1 answer from 2 or more possibilities. Arc tangent for example. The arc tangent of 1 is returned as 45 degrees, but it could also be 225 or even 405. As with many things in life and programming there is a convention we know and can rely on. Square root functions return positive values is one of them. It is up to us, the programmers, to keep in mind there are other solutions and to act on them if needed in code.
By the way this is a common issue in robotics when dealing with kinematics and inverse kinematics equations where there are multiple solutions of links positions corresponding to Cartesian positions.
In mathematics, by convention it's always assumed that you want the positive square root of something unless you explicitly say otherwise. The square root of four really is two. If you want the negative answer, put a negative sign in front. If you want both, put the plus-or-minus sign. Without this convention it would be impossible to write equations; you would never know what the person intended even if they did put a sign in front (because it could be the negative of the negative square root, for example). Also, how exactly would you write any kind of computer code involving mathematics if operators started returning two values? It would break everything.
The unfortunate exception to this convention is when solving for variables. In the following equation:
x^2 = 4
You have no choice but to consider both possible values for X. if you take the square root of both sides, you get x = 2 but now you must put in the plus or minus sign to make sure you aren't missing any possible solutions. Also, remember that in this case it's technically X that can be either plus or minus, not the square root of four.
Because multiple return types are annoying to implement. If you really need the other result, isn't it easy enough to just multiple the result by -1?
Because most programmers only want one answer.
It's easy enough to generate the negative value from the positive value if the caller wants it. For most code the caller only uses the positive value.
However, nowadays it's easy to return two values in many languages. In JavaScript:
var sqrts=function(x) {
var s=Math.sqrt(x);
if (s>0) {
return [s,-s];
} else {
return [0];
}
}
As long as the caller knows to iterate through the array that comes back, you're gold.
>sqrts(2)
[1.4142135623730951, -1.4142135623730951]
I think because the function is called "sqrt", and if you wanted multiple roots, you would have to call the function "sqrts", which doesn't exist, so you can't do it.
The more serious answer is that you're suggesting a specific instance of a larger issue. Many equations, and commonly inverse functions (including sqrt) have multiple possible solutions, such as arcsin, etc, and these are, in general, an issue. With arcsin, for example, should one return an infinite number of answers? See, for example, discussions about branch cuts.
Because it was historically defined{{citation needed}} as the function which gives the side length of a square of known surface. And length is positive in that context.
you can always tell what is the other number, so maybe it's not necessary to return both of them.
It's likely because when people use a calculator to figure out a square root, they only want the positive value.
Go one step further and ask why your calculator won't let you take the square root of a negative number. It's possible, using imaginary numbers, but the average user has absolutely zero use for this.
On imaginary numbers.