This is related to another question I have except there I am looking for a vectorized/broadcasting solution.
Here I am hoping to try using np.fromfunction but am having issues.
The MWE is
# L1 and L2 will range from 0 to 3 typically, sometimes up to 5
# all of the following are dummy values but match correct `type`
L1, L2, x1, x2, fac = 2, 3, 2.0, 4.5, 2.3
saved_values = np.random.uniform(high=75.0, size=[max(L1,L2) + 1, max(L1,L2) + 1])
facts = np.random.uniform(high=65.0, size=[L1 + L2 + 1])
val = 0
for i in range(L1+1):
sf = saved_values[L1][i] * x1 ** (L1 - i)
for j in range(L2 + 1):
m = i + j
if m % 2 == 0:
num = sf * facts[m] / (2 * fac) ** (m / 2)
val += saved_values[L2][j] * x1 ** (L1 - j) * num
rather than use a double for loop, which is not vectorized and potentially slower (being vectorized is more important for me), I would like to use np.fromfunction to fill a matrix with the necessary values
I tried the following
matrix = np.fromfunction(lambda i, j:
saved_values[L1][i] * x1 ** (L1 - i) *
saved_values[L2][j] * x2 ** (L2 - j) *
facts[i+j] / (2 * fac) **( (i+j) / 2),
(L1+1,L2+1)
)
This generates the double for loop matrix, I then need to go over it and make anything that would fail (i+j) % 2 == 0 have a zero value, and then sum all elements in the matrix.
However
I get the following error
Traceback (most recent call last):
File "main.py", line 54, in <module>
(L1+1,L2+1)
File "/usr/lib/python3/dist-packages/numpy/core/numeric.py", line 1808, in fromfunction
return function(*args,**kwargs)
File "main.py", line 53, in <lambda>
facts[i+j] / (2 * fac) **( (i+j) / 2),
IndexError: arrays used as indices must be of integer (or boolean) type
As I suppose, you think that the function passed as the first parameter
to np.fromfunction is called several times, for each combination
of i and j.
Unfortunately, this is not the case. To see the actual way how
it operates, make such experiment:
def myFun(i, j):
print(f'myFun:\n{i.shape}\n{i}\n{j.shape}\n{j}')
rv = i + j
return rv
I deliberately put a trace printout, to show what I mean.
Then run it:
np.fromfunction(myFun, (2, 3))
The result is:
myFun:
(2, 3)
[[0. 0. 0.]
[1. 1. 1.]]
(2, 3)
[[0. 1. 2.]
[0. 1. 2.]]
array([[0., 1., 2.],
[1., 2., 3.]])
The first part is the printout, occuring only once and the second - the
actual result. Note that:
shapes of both i and j are (2,3),
both i and j are actually arrays, not integers,
the whole computation is performed by addition of both arrays.
So the conclusions are that:
the passed function operates in a vectorized way,
on arrays of parameters, adding respective elements and
returning the result,
your concept is probably impossible to implement.
Related
I need to solve a non linear optimization problem in Python. I found out that scipy solves optimization problems, however I don't know what I am doing wrong since with some example input it can't find the correct solution that I have in NEOS server solver Knitro AMPL.
My problem is that, given a set of points it must find the biggest ellipse inscribed that at max touches those points and the points are never included inside of it.
Theory
Formulating the optimization problem, I have a and b the semiaxis, phi the rotation, xc and yc the coordinates of the centre and points the list of points with each element in the form of [x, y] -> [0, 1] indices.
On paper the problem and the constraints are these, a, b, phi, xc, yc are real, the points are integers:
NEOS
The files I used in NEOS are these:
mod
dat
run
With successful results (complete):
xc = 143.012
yc = 262.634
a = 181.489
b = 140.429
phi = 1.43575
Python
So, my python code is this, it is my first time using scipy for optimization, so I don't exclude errors of understanding how it works from the documentation.
from typing import List
import numpy as np
from scipy.optimize import *
def ellipse_calc(
points: List[List[int]],
verbose: bool = False
):
centre = [0, 0]
for i in range(len(points)):
centre[0] += points[i][0]
centre[1] += points[i][1]
centre[0] /= len(points)
centre[1] /= len(points)
if verbose:
print(f'centre: {centre[0]:.2f}, {centre[1]:.2f}')
max_x = max([p[0] for p in points])
max_y = max([p[1] for p in points])
min_x = min([p[0] for p in points])
min_y = min([p[1] for p in points])
initial_axis = 0.25 * (max_x - min_x + max_y - min_y)
if verbose:
print(initial_axis)
constraints = [
NonlinearConstraint(lambda x: x[0], 1, np.inf),
NonlinearConstraint(lambda x: x[1], 1, np.inf),
NonlinearConstraint(lambda x: x[2], 0, np.inf),
]
for i in range(len(points)):
constraints += [NonlinearConstraint(
lambda x:
(points[i][0] - x[3]) ** 2 * (np.cos(x[2]) ** 2 / x[0]**2 + np.sin(x[2]) ** 2 / x[1]**2) +
(points[i][1] - x[4]) ** 2 * (np.sin(x[2]) ** 2 / x[0]**2 + np.cos(x[2]) ** 2 / x[1]**2) +
2 * (points[i][0] - x[3]) * (points[i][1] - x[4]) *
np.cos(x[2]) * np.sin(x[2]) * (1 / x[1]**2 - 1 / x[0]**2), 1, np.inf)]
result = minimize(
lambda x: -np.pi * x[0] * x[1],
[initial_axis, initial_axis, 0, centre[0], centre[1]],
constraints=constraints
)
print(result)
if __name__ == '__main__':
points = [[50,44],[91,44],[161,44],[177,44],[44,88],[189,88],[239,88],[259,88],[2,132],[250,132],[2,176],[329,176],[2,220],[289,220],[2,264],[288,264],[2,308],[277,308],[2,352],[285,352],[2,396],[25,396],[35,396],[231,396],[284,396],[298,396],[36,440],[76,440],[106,440],[173,440]]
ellipse_calc(points, True)
This try, that has the same data I tried on NEOS gives as output the following:
fun: -8.992626773255127e+40
jac: array([-5.68832805e+20, -4.96651566e+20, -0.00000000e+00, -0.00000000e+00,
-0.00000000e+00])
message: 'Inequality constraints incompatible'
nfev: 54
nit: 10
njev: 9
status: 4
success: False
x: array([ 1.58089104e+20, 1.81065104e+20, -1.24564497e+15, -1.55647883e+10,
-2.76654483e+10])
Does anyone know what I am doing wrong and how to fix it? Also, I don't really know if it is possible to solve this problem with scipy, in that case I am looking for a free library to solve it or even to alternative methods of finding that ellipse equation
This isn't a complete answer, but it should help you to get started. Here are two hints:
Pass simple box constraints on the variables as boundaries, not as constraints. That is, use
bounds = [(1, None), (1, None), (0, None), (None, None), (None, None)]
and pass it to minimize via the bounds parameter.
You need to be really careful when defining constraints through lambda expressions inside a loop, see here. You need to capture the loop variable i by lambda x, i=i: your_fun. Otherwise, each of your constraints uses i=29 and thus evaluates the last point. This can easily be observed by evaluating all constraints for a specific value.
Then you should at least get a feasible solution with an objective value of 79384. Note also that you can shorten your code significantly by using numpy functions instead of loops.
I am trying to train a multi-class classifier with multinomial logistic regression and gradient descent. Specifically, the model will have a trained weights matrix w with shape (C, D) where C is the number of classes and D is the number of features of each input. Also, we will have a bias vector b with dimension (C,). We have an (N, D) input matrix X, where N is the number of training inputs, and a vector y with shape (N,), where each entry in y is a number from 0 to C - 1, indicating which class the input belongs to. I have written the following code:
for _ in range(max_iterations):
z = np.apply_along_axis(lambda v: v - max(v), 1, X # w.T + b)
probs = np.exp(z)
denom = np.sum(probs, axis=1)
for i in range(C):
for j in range(N):
if i == y[j]:
w[i] -= (step_size / N) * ((probs[j][i] / denom[j]) - 1) * X[j]
b[i] -= (step_size / N) * ((probs[j][i] / denom[j]) - 1)
else:
w[i] -= (step_size / N) * (probs[j][i] / denom[j]) * X[j]
b[i] -= (step_size / N) * (probs[j][i] / denom[j])
This produces the correct weights and bias that I want, but clearly it doesn't take advantage of numpy's operations to speed things up. So I tried to speed some of it up with the following code:
for _ in range(max_iterations):
z = np.apply_along_axis(lambda v: v - max(v), 1, X # w.T + b)
probs = np.exp(z)
denom = np.sum(probs, axis=1)
s = np.zeros((N, C))
for i in range(N):
s[i] = probs[i] / denom[i]
for i in range(N):
s[i][y[i]] += -1
for c in range(C):
grad_w = s.T[c] # X
w[c] += (step_size / N) * grad_w
b[c] += (step_size / N) * sum(s.T[c])
I was hoping that this would produce the same results as in the previous part while being faster... and it managed to be faster, but with incorrect results.
So I have a couple of questions. First, why is my second piece of code not producing the right results, and what would be a fix for it? Second, and more importantly, how would I optimize this further? This is mainly for me to learn how to take advantage of numpy's vectorized operations.
This may help with some of the iterations.
Start with a small 2d array:
In [251]: probs = np.arange(12).reshape(3,4)
In [252]: denom = np.sum(probs, axis=1)
In [253]: denom
Out[253]: array([ 6, 22, 38])
To divide a (3,4) array by a (3,), we need to make the later (3,1):
In [254]: probs/denom[:,None]
Out[254]:
array([[0. , 0.16666667, 0.33333333, 0.5 ],
[0.18181818, 0.22727273, 0.27272727, 0.31818182],
[0.21052632, 0.23684211, 0.26315789, 0.28947368]])
Read, and reread, the numpy documentation on broadcasting if that doesn't make sense.
Another way to get the required 2d denom, is:
In [255]: denom = np.sum(probs, axis=1, keepdims=True)
In [256]: denom
Out[256]:
array([[ 6],
[22],
[38]])
In [257]: probs/denom
Out[257]:
array([[0. , 0.16666667, 0.33333333, 0.5 ],
[0.18181818, 0.22727273, 0.27272727, 0.31818182],
[0.21052632, 0.23684211, 0.26315789, 0.28947368]])
The same should work for the max subtraction that you use with apply_along_axis. apply... is not a speed tool, and not superior to simple iteration.
In [258]: np.max(probs, axis=1, keepdims=True)
Out[258]:
array([[ 3],
[ 7],
[11]])
In [259]: probs - _
Out[259]:
array([[-3, -2, -1, 0],
[-3, -2, -1, 0],
[-3, -2, -1, 0]])
I am not very experienced with sympy so sorry if this is a simple question.
How can I use sympy to expand binomial expressions? For example say I want to have sympy compute the coefficient of $x^2$ in the polynomial $(x^2 + x + 1)^n$ (where I would expect the answer to be $n + \binom{n}{2}$).
I tried the following code:
x = symbols('x')
n = symbols('n', integer=True, nonnegative = True)
expand((x**2+x+1)**n)
but the result is just $(x^2+x+1)^n$ whereas I would want the binomial expansion, i.e .
Thanks in advance.
If the exponent is not symbolic then the following gives the coefficient very quickly for the power of an arbitrary polynomial with integer coefficients, e.g.,
>>> eq
x**3 + 3*x + 2
>>> (Poly(eq)**42).coeff_monomial(x**57)
2294988464559317378977138572972
But there is currently no routine to indicate the coefficient if the exponent of the polynomial is symbolic. rsolve can also be used to express the closed form if a pattern can be seen in the coefficient, too:
>>> print([((x**2+x+1)**i).expand().coeff(x**2) for i in range(8)])
[0, 1, 3, 6, 10, 15, 21, 28]
>>> from sympy.abc import n
>>> f=Function('f') # f(n) represents the coefficient of x**2 for a given n
The coefficient for x^2 for a given n is n more than the last value:
>>> rsolve(f(n)-f(n-1)-n, f(n),{f(0):0,f(1):1})
n*(n + 1)/2
This final expression is the coefficient of x^2 for arbitrary n.
Issue 17889 gives a routine that will compute the coefficient of a term in a univariate polynomial (with arbitrary coefficients for each term) raised to the power of n:
>>> eq = 2 + x + x**2
>>> unicoeff(eq, 4).simplify()
Piecewise(
(0, n < 2),
(2**(n - 3)*n*(n - 1), n < 3),
(2**(n - 4)*n**2*(n - 1), n < 4),
(2**n*n*(n - 1)*(n**2 + 19*n + 6)/384, True))
>>> _.subs(n, 5)
210
>>> (eq**5).expand().coeff(x**4)
210
For your expression (where the constant is 1):
>>> unicoeff(1+x+x**2,2).simplify()
Piecewise((0, n < 1), (n, n < 2), (n*(n + 1)/2, True))
I have a bumpy array. I want to find the number of points which lies within an epsilon distance from each point.
My current code is (for a n*2 array, but in general I expect the array to be n * m)
epsilon = np.array([0.5, 0.5])
np.array([ 1/np.float(np.sum(np.all(np.abs(X-x) <= epsilon, axis=1))) for x in X])
But this code might not be efficient when it comes to an array of let us say 1 million rows and 50 columns. Is there a better and more efficient method ?
For example data
X = np.random.rand(10, 2)
you can solve this using broadcasting:
1 / np.sum(np.all(np.abs(X[:, None, ...] - X[None, ...]) <= epsilon, axis=-1), axis=-1)
I want to construct a multivariate Normal model in PyMC3 in which the mean value and precision matrix involve probabilistic variables. h is meant to act as a latent variable in an larger project to which this code snippet belongs.
When I run the code provided below, I get the error message shown, and I'm not sure exactly how to interpret it. As far as I can see, the dimension of the mean value of the MvNormal (2-row column vector) match the dimension of the precision matrix B (2 x 2 matrix), so I don't expect it's the dimensions of these objects that are causing the problem. I don't know what other variables could be causing some error related to dimensions to be thrown up though. Can anyone shed some light on this please?
Here is the code:
import pymc3 as pm
import theano.tensor as tt
with pm.Model() as model:
# A matrix
a1 = pm.Uniform('a1', 0., 1.)
a2 = pm.Uniform('a2', 0., 1.)
ix = ([0, 0, 1, 1], [0, 1, 0, 1])
A = tt.eye(2)
A = tt.set_subtensor(A[ix], [a1, a2, 1, 0])
# B matrix
b1 = pm.Uniform('b1', 0., 1.)
b2 = pm.Uniform('b2', 0., 1.)
ix = ([0, 1], [0, 1])
B = tt.eye(2)
B = tt.set_subtensor(B[ix], [b1 ** 2, b2 ** 2])
# Model
y0 = pm.Normal('y0', mu=0., sd=1., observed=0)
y1 = pm.Normal('y1', mu=1., sd=1., observed=1)
s_v = tt.stack([y1, y0]).T
h = pm.MvNormal("h", mu=pm.math.dot(A, s_v), tau=B)
Error message:
h = pm.MvNormal("h", mu=pm.math.dot(A, s_v), tau=B)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/distributions/distribution.py", line 42, in __new__
return model.Var(name, dist, data, total_size)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/model.py", line 809, in Var
total_size=total_size, model=self)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/model.py", line 1209, in __init__
self.logp_elemwiset = distribution.logp(self)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/distributions/multivariate.py", line 274, in logp
quaddist, logdet, ok = self._quaddist(value)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/distributions/multivariate.py", line 85, in _quaddist
raise ValueError('Invalid dimension for value: %s' % value.ndim)
ValueError: Invalid dimension for value: 0```
I believe that you are missing the "shape" argument in the pm.MvNormal call, which lets it handle the right size of values. For example, if you have 7 variables, set shape=7.