I'm trying to create Skewed Normal distribution with the following PDF
I'm using the following command for that (referenced from http://en.wikipedia.org/wiki/Skew_normal_distribution):
I'm trying to do the following:
SkewedNormal := Distribution(PDF = unapply(2*phi(x, mu, sigma)*Phi(alpha*x, mu, sigma), x, mu, sigma, alpha))
This command executes without errors, the same as the following command:
R := RandomVariable(SkewNormal)
but the problems start when I try to do the following:
CDF(R,x)
Error, (in Statistics:-CDF) invalid input: q uses a 3rd argument, sigma, which is missing
Ok, I add the third parameter:
CDF(R,x,y)
Error, (in Statistics:-CDF) unexpected parameters: y
If you try previously to init random variable the following way:
R := RandomVariable(SkewNormal(mu, sigma))
Error, (in Statistics:-Distribution) invalid input: IsKnownDistribution expects its 1st argument, dn, to be of type
name, but received module () export Conditions, PDF, Type; option Distribution, Continuous; end module
How do you create parametric distribution in Maple 14?
Can you not unapply with respect to only x? (Note you had a typo in the posted code, using SkewedNormal vs SkewNormal.)
with(Statistics):
SkewNormal := Distribution(PDF =
unapply(2*phi(x, mu, sigma)*Phi(alpha*x, mu, sigma), x));
R := RandomVariable(SkewNormal):
CDF(R,x);
The final result there is an expression containing alpha, mu, and sigma. So subs or eval could then be used to instantiate at values for the parameters.
In case anybody will face the same problem here's how I managed to solve it this way:
SkewedNormal := (xi, omega, alpha) ->
Distribution
(
PDF = ((x) -> x*sqrt(2)*exp(-(1/2)*(x-xi)^2/omega^2)*(1/2+(1/2)*erf((1/2)*alpha*(x-xi)*sqrt(2)/omega))/(omega*sqrt(Pi))),
CDF = (proc (x) local t; options operator, arrow; return 1/2+(1/2)*erf((1/2)*(x-xi)*sqrt(2)/omega)-(int(exp(-(1/2)*(t-xi)^2*(1+t^2)/omega^2)/(1+t^2), t = 0 .. alpha))/Pi end proc),
Mean = xi+omega*alpha*sqrt(2/Pi)/sqrt(1+alpha^2),
Variance = omega^2*(1-2*alpha^2/(sqrt(1+alpha^2)^2*Pi)),
MGF = ((x) -> 2*exp(xi*x+(1/2)*omega^2*x^2)*(1/2+(1/2)*erf((1/2)*omega*alpha*x*sqrt(2)/sqrt(1+alpha^2))))
)
This way allows defining parametric distribution
Examples:
X:=SkewedNormal(u,v,m); # Skewed normal distribution with xi=u, omega=v, alpha=m
Y:=SkewedNormal(a,b,c); # Skewed normal distribution with xi=a, omega=b, alpha=c
It also works with functions from Statistics package, such as RandomVariable:
Rx:=RandomVariable(X);
Ry:=RandomVariable(Y);
And calling:
CDF(Ry,x);
Gives
Related
I'm attempting to solve the differential equation:
m(t) = M(x)x'' + C(x, x') + B x'
where x and x' are vectors with 2 entries representing the angles and angular velocity in a dynamical system. M(x) is a 2x2 matrix that is a function of the components of theta, C is a 2x1 vector that is a function of theta and theta' and B is a 2x2 matrix of constants. m(t) is a 2*1001 array containing the torques applied to each of the two joints at the 1001 time steps and I would like to calculate the evolution of the angles as a function of those 1001 time steps.
I've transformed it to standard form such that :
x'' = M(x)^-1 (m(t) - C(x, x') - B x')
Then substituting y_1 = x and y_2 = x' gives the first order linear system of equations:
y_2 = y_1'
y_2' = M(y_1)^-1 (m(t) - C(y_1, y_2) - B y_2)
(I've used theta and phi in my code for x and y)
def joint_angles(theta_array, t, torques, B):
phi_1 = np.array([theta_array[0], theta_array[1]])
phi_2 = np.array([theta_array[2], theta_array[3]])
def M_func(phi):
M = np.array([[a_1+2.*a_2*np.cos(phi[1]), a_3+a_2*np.cos(phi[1])],[a_3+a_2*np.cos(phi[1]), a_3]])
return np.linalg.inv(M)
def C_func(phi, phi_dot):
return a_2 * np.sin(phi[1]) * np.array([-phi_dot[1] * (2. * phi_dot[0] + phi_dot[1]), phi_dot[0]**2])
dphi_2dt = M_func(phi_1) # (torques[:, t] - C_func(phi_1, phi_2) - B # phi_2)
return dphi_2dt, phi_2
t = np.linspace(0,1,1001)
initial = theta_init[0], theta_init[1], dtheta_init[0], dtheta_init[1]
x = odeint(joint_angles, initial, t, args = (torque_array, B))
I get the error that I cannot index into torques using the t array, which makes perfect sense, however I am not sure how to have it use the current value of the torques at each time step.
I also tried putting odeint command in a for loop and only evaluating it at one time step at a time, using the solution of the function as the initial conditions for the next loop, however the function simply returned the initial conditions, meaning every loop was identical. This leads me to suspect I've made a mistake in my implementation of the standard form but I can't work out what it is. It would be preferable however to not have to call the odeint solver in a for loop every time, and rather do it all as one.
If helpful, my initial conditions and constant values are:
theta_init = np.array([10*np.pi/180, 143.54*np.pi/180])
dtheta_init = np.array([0, 0])
L_1 = 0.3
L_2 = 0.33
I_1 = 0.025
I_2 = 0.045
M_1 = 1.4
M_2 = 1.0
D_2 = 0.16
a_1 = I_1+I_2+M_2*(L_1**2)
a_2 = M_2*L_1*D_2
a_3 = I_2
Thanks for helping!
The solver uses an internal stepping that is problem adapted. The given time list is a list of points where the internal solution gets interpolated for output samples. The internal and external time lists are in no way related, the internal list only depends on the given tolerances.
There is no actual natural relation between array indices and sample times.
The translation of a given time into an index and construction of a sample value from the surrounding table entries is called interpolation (by a piecewise polynomial function).
Torque as a physical phenomenon is at least continuous, a piecewise linear interpolation is the easiest way to transform the given function value table into an actual continuous function. Of course one also needs the time array.
So use numpy.interp1d or the more advanced routines of scipy.interpolate to define the torque function that can be evaluated at arbitrary times as demanded by the solver and its integration method.
I am still struggling with scipy.integrate.quad.
Sparing all the details, I have an integral to evaluate. The function is of the form of the integral of a product of functions in x, like so:
Z(k) = f(x) g(k/x) / abs(x)
I know for certain the range of integration is between tow positive numbers. Oddly, when I pick a wide range that I know must contain all values of x that are positive - like integrating from 1 to 10,000,000 - it intgrates fast and gives an answer which looks right. But when I fingure out the exact limits - which I know sice f(x) is zero over a lot of the real line - and use those, I get another answer that is different. They aren't very different, though I know the second is more accurate.
After much fiddling I got it to work OK, but then needed to add in an exopnentiation - I was at least getting a 'smooth' answer for the computed function of z. I had this working in an OK way before I added in the exponentiation (which is needed), but now the function that gets generated (z) becomes more and more oscillatory and peculiar.
Any idea what is happening here? I know this code comes from an old Fortran library, so there must be some known issues, but I can't find references.
Here is the core code:
def normal(x, mu, sigma) :
return (1.0/((2.0*3.14159*sigma**2)**0.5)*exp(-(x-
mu)**2/(2*sigma**2)))
def integrand(x, z, mu, sigma, f) :
return np.exp(normal(z/x, mu, sigma)) * getP(x, f._x, f._y) / abs(x)
for _z in range (int(z_min), int(z_max) + 1, 1000):
z.append(_z)
pResult = quad(integrand, lb, ub,
args=(float(_z), MU-SIGMA**2/2, SIGMA, X),
points = [100000.0],
epsabs = 1, epsrel = .01) # drop error estimate of tuple
p.append(pResult[0]) # drop error estimate of tuple
By the way, getP() returns a linearly interpolated, piecewise continuous,but non-smooth function to give the integrator values that smoothly fit between the discrete 'buckets' of the histogram.
As with many numerical methods, it can be very sensitive to asymptotes, zeros, etc. The only choice is to keep giving it 'hints' if it will accept them.
Hi I am working with both the SLSQP solver on python and diffev2 which is part of the mystic package.
For multiple parameters the format shown below works:
bnds = ((0, 1e3), (0, 1e-4))
optimize.minimize(Error, [1e-8, 1e-7], args=(E, Bt, L, dt, TotT, nz, Co, Exmatrix), method = 'SLSQP', bounds = bnds)
I want to optimize only one parameter and that is when I run into the error: SLSQP Error: the length of bounds is not compatible with that of x0.
I use the syntax shown below:
bnds = ((1e-9, 1e-2))
optimize.minimize(Error, [1e-8], args=(U, E, Bt, L, dt, TotT, nz, Co, Exmatrix), method = 'SLSQP', bounds = bnds)
I am not sure what is wrong, as I have only one tuple pair in the bnds variable and one guess, not sure what is wrong.
Straight from python's docs:
The trailing comma is required only to create a single tuple (a.k.a. a singleton); it is optional in all other cases. A single expression without a trailing comma doesn’t create a tuple, but rather yields the value of that expression. (To create an empty tuple, use an empty pair of parentheses: ().)
Working alternatives:
bnds = ((1e-9, 1e-2),)
bnds = [(1e-9, 1e-2)]
Internally, this happens:
bnds = ((1e-9, 1e-2))
np.array(bnds, float).shape
# (2,)
bnds = ((1e-9, 1e-2),)
np.array(bnds, float).shape
# (1, 2)
# and then N is compared to the size of the first dimension (2 vs. 1)
(And make sure you got a reason not to use minimize_scalar)
In the theano derivatives tutorial here:
http://deeplearning.net/software/theano/tutorial/gradients.html#tutcomputinggrads
the example of Lop works without an explicit value of the W matrix in the dot product. And, in fact, the partial derivatives in this case do remove the values of the components of W so they are not needed.
But, attempting a similar thing with the Rop throws an error:
theano.gof.fg.MissingInputError: ("An input of the graph, used to compute dot(Elemwise{second,no_inplace}.0, ), was not provided and not given a value.
How is this different?
Theano will try to optimize the computation graph, but it does not always work.
In the Lop example, Theano can detect that we don't actually need that W, but when changed to the Rop it just can't.
The Lop example:
W = T.dmatrix('W')
v = T.dvector('v')
x = T.dvector('x')
y = T.dot(x, W)
VJ = T.Lop(y, W, v)
f = theano.function([v, x], VJ)
f([2, 2], [0, 1])
If I just change y = T.dot(x, W) to y = T.dot(x, W**1), Theano will fail to do the optimization and throw the same error message at me say that I did not provide enough parameters.
Actually in the Rop example, if we change the values given to W, it does not affect the result at all, because Theano failed to optimize that.
p.s. I find the Theano documents very unclear sometimes.
Problem Synopsis:
When attempting to use the scipy.optimize.fmin_bfgs minimization (optimization) function, the function throws a
derphi0 = np.dot(gfk, pk)
ValueError: matrices are not aligned
error. According to my error checking this occurs at the very end of the first iteration through fmin_bfgs--just before any values are returned or any calls to callback.
Configuration:
Windows Vista
Python 3.2.2
SciPy 0.10
IDE = Eclipse with PyDev
Detailed Description:
I am using the scipy.optimize.fmin_bfgs to minimize the cost of a simple logistic regression implementation (converting from Octave to Python/SciPy). Basically, the cost function is named cost_arr function and the gradient descent is in gradient_descent_arr function.
I have manually tested and fully verified that *cost_arr* and *gradient_descent_arr* work properly and return all values properly. I also tested to verify that the proper parameters are passed to the *fmin_bfgs* function. Nevertheless, when run, I get the ValueError: matrices are not aligned. According to the source review, the exact error occurs in the
def line_search_wolfe1
function in # Minpack's Wolfe line and scalar searches as supplied by the scipy packages.
Notably, if I use scipy.optimize.fmin instead, the fmin function runs to completion.
Exact Error:
File
"D:\Users\Shannon\Programming\Eclipse\workspace\SBML\sbml\LogisticRegression.py",
line 395, in fminunc_opt
optcost = scipy.optimize.fmin_bfgs(self.cost_arr, initialtheta, fprime=self.gradient_descent_arr, args=myargs, maxiter=maxnumit, callback=self.callback_fmin_bfgs, retall=True)
File
"C:\Python32x32\lib\site-packages\scipy\optimize\optimize.py", line
533, in fmin_bfgs old_fval,old_old_fval)
File "C:\Python32x32\lib\site-packages\scipy\optimize\linesearch.py", line
76, in line_search_wolfe1
derphi0 = np.dot(gfk, pk)
ValueError: matrices are not aligned
I call the optimization function with:
optcost = scipy.optimize.fmin_bfgs(self.cost_arr, initialtheta, fprime=self.gradient_descent_arr, args=myargs, maxiter=maxnumit, callback=self.callback_fmin_bfgs, retall=True)
I have spent a few days trying to fix this and cannot seem to determine what is causing the matrices are not aligned error.
ADDENDUM: 2012-01-08
I worked with this a lot more and seem to have narrowed the issues (but am baffled on how to fix them). First, fmin (using just fmin) works using these functions--cost, gradient. Second, the cost and the gradient functions both accurately return expected values when tested in a single iteration in a manual implementation (NOT using fmin_bfgs). Third, I added error code to optimize.linsearch and the error seems to be thrown at def line_search_wolfe1 in line: derphi0 = np.dot(gfk, pk).
Here, according to my tests, scipy.optimize.optimize pk = [[ 12.00921659]
[ 11.26284221]]pk type = and scipy.optimize.optimizegfk = [[-12.00921659] [-11.26284221]]gfk type =
Note: according to my tests, the error is thrown on the very first iteration through fmin_bfgs (i.e., fmin_bfgs never even completes a single iteration or update).
I appreciate ANY guidance or insights.
My Code Below (logging, documentation removed):
Assume theta = 2x1 ndarray (Actual: theta Info Size=(2, 1) Type = )
Assume X = 100x2 ndarray (Actual: X Info Size=(2, 100) Type = )
Assume y = 100x1 ndarray (Actual: y Info Size=(100, 1) Type = )
def cost_arr(self, theta, X, y):
theta = scipy.resize(theta,(2,1))
m = scipy.shape(X)
m = 1 / m[1] # Use m[1] because this is the length of X
logging.info(__name__ + "cost_arr reports m = " + str(m))
z = scipy.dot(theta.T, X) # Must transpose the vector theta
hypthetax = self.sigmoid(z)
yones = scipy.ones(scipy.shape(y))
hypthetaxones = scipy.ones(scipy.shape(hypthetax))
costright = scipy.dot((yones - y).T, ((scipy.log(hypthetaxones - hypthetax)).T))
costleft = scipy.dot((-1 * y).T, ((scipy.log(hypthetax)).T))
def gradient_descent_arr(self, theta, X, y):
theta = scipy.resize(theta,(2,1))
m = scipy.shape(X)
m = 1 / m[1] # Use m[1] because this is the length of X
x = scipy.dot(theta.T, X) # Must transpose the vector theta
sig = self.sigmoid(x)
sig = sig.T - y
grad = scipy.dot(X,sig)
grad = m * grad
return grad
def fminunc_opt_bfgs(self, initialtheta, X, y, maxnumit):
myargs= (X,y)
optcost = scipy.optimize.fmin_bfgs(self.cost_arr, initialtheta, fprime=self.gradient_descent_arr, args=myargs, maxiter=maxnumit, retall=True, full_output=True)
return optcost
In case anyone else encounters this problem ....
1) ERROR 1: As noted in the comments, I incorrectly returned the value from my gradient as a multidimensional array (m,n) or (m,1). fmin_bfgs seems to require a 1d array output from the gradient (that is, you must return a (m,) array and NOT a (m,1) array. Use scipy.shape(myarray) to check the dimensions if you are unsure of the return value.
The fix involved adding:
grad = numpy.ndarray.flatten(grad)
just before returning the gradient from your gradient function. This "flattens" the array from (m,1) to (m,). fmin_bfgs can take this as input.
2) ERROR 2: Remember, the fmin_bfgs seems to work with NONlinear functions. In my case, the sample that I was initially working with was a LINEAR function. This appears to explain some of the anomalous results even after the flatten fix mentioned above. For LINEAR functions, fmin, rather than fmin_bfgs, may work better.
QED
As of current scipy version you need not pass fprime argument. It will compute the gradient for you without any issues. You can also use 'minimize' fn and pass method as 'bfgs' instead without providing gradient as argument.