If I have a cost that is the imaginary part of a complex number, trying to obtain the gradient with theano I get the following error:
TypeError: Elemwise{imag,no_inplace}.grad illegally returned an integer-valued variable. (Input index 0, dtype complex128)
Is it not possible to use the imaginary part as cost despite it being a real-valued cost?
Edit. Minimal working example
import theano.tensor as T
from theano import function
a = T.zscalar('a')
f = function([a], T.grad(T.imag(a),a))
I would expect this to work as T.imag(a) is a real scalar cost..
Related
I use torch.normal to generate (1,n)-dimensional samples with an std vector that includes zero values. This doesn't generate any errors (when std[k] is zero the corresponding mean[k] is sampled).
However, torch.distributions.MultivariateNormal doesn't accept diagonal covariance matrix whose diagonal values are std[k]**2
I need the MultivariateNormal object to compute log_prob of the generated sample for which i used torch.normal
Code:
import torch
mean = torch.Tensor([1.00000,1.00000,1.00000,1.00000,1.00000,-1.00000,-1.00000,-1.00000,-1.00000])
std = torch.Tensor([17708.14062,68.16734,0.00000,0.00000,5917.79932,15390.00488, 0.00000,10070.79395,4994.94434])
x = torch.normal(mean, std)
torch.distributions.MultivariateNormal(loc=mean, covariance_matrix=torch.Tensor(np.diag(np.array(std)**2))).log_prob(x)
Error:
ValueError: Expected parameter covariance_matrix (Tensor of shape (9, 9)) of distribution MultivariateNormal(loc: torch.Size([9]), covariance_matrix: torch.Size([9, 9])) to satisfy the constraint PositiveDefinite(), but found invalid values
I don't get why torch.normal doesn't mind zero values in the std but normal.distributions.MultivariateNormal does.
I tried to look up special parameters that can be passed to the constructor to ignore this, like the 'allow_singular' parameter in the scipy library, but there seems to be none.
from scipy.stats import multivariate_normal
pi = multivariate_normal(mean=mean, cov=np.diag(np.array(std)**2), allow_singular=True).pdf(
np.array(action))
I'm using preprocessing from package sklearn to normalize data as follows:
import pandas as pd
import urllib3
from sklearn import preprocessing
decathlon = pd.read_csv("https://raw.githubusercontent.com/leanhdung1994/Deep-Learning/main/decathlon.txt", sep='\t')
decathlon.describe()
nor_df = decathlon.copy()
nor_df.iloc[:, 0:10] = preprocessing.scale(decathlon.iloc[:, 0:10])
nor_df.describe()
The result is
The mean is -1.516402e-16, which is almost 0. On the contrary, the variance is 1.012423e+00, which is 1.012423. For me, 1.012423 is not considered as near 1.
Could you please elaborate on this phenomenon?
In this instance sklearn and pandas calculate std differently.
sklearn.preprocessing.scale:
We use a biased estimator for the standard deviation, equivalent to
numpy.std(x, ddof=0). Note that the choice of ddof is unlikely to
affect model performance.
pandas.Dataframe.describe uses pandas.core.series.Series.std where:
Normalized by N-1 by default. This can be changed using the ddof argument
...
ddof : int, default 1
Delta Degrees of Freedom. The divisor used in calculations is N - ddof,
where N represents the number of elements.
It should be noted that, in 2020-10-28, pandas.Dataframe.describe does not have a ddof parameter so the default of ddof=1 is always used for Series.
I am trying to generate a number from the normal distribution using:
from torch.distributions import Normal
noise = Normal(th.tensor([0]), th.tensor(3.20))
noise = noise.sample()
But I am getting this error: RuntimeError: _th_normal not supported on CPUType for Long
Your first tensor th.tensor([0]) is of torch.Long type due to automatic type inference from passed value while float or FloatTensor is required by function.
You can solve it by passing 0.0 explicitly like this:
import torch
noise = torch.distributions.Normal(torch.tensor([0.0]), torch.tensor(3.20))
noise = noise.sample()
Better yet, drop torch.tensor altogether, in this case Python types will be automatically casted to float if possible, so this is valid as well:
import torch
noise = torch.distributions.Normal(0, 3.20)
noise = noise.sample()
And please, do not alias torch as th, it is not official, use fully qualified name as this only confuses everyone.
I am trying to create a Fibonacci number with high n by using matrix exponential but it gives me negative result. I have tried to change the integer objects but failed.
import numpy as np
def matrixmul(a,n):
a=np.array([[1,1],[1,0]])
return ((np.array([1,1],[1,0], dtype=np.object))**n)
matrixMul(a,100)
my output is
array([[-1869596475, -980107325],
[ -980107325, -889489150]])
but it was wrong. there should not be any negative number.
It's hard to answer on your question. Your code have some bugs:
You haven't initialize a
Also name of defined function is different than used (python is case-sensitive)
Then in function you are not using a (because it is not in return)
And most important thing is that you can not use **n too get exponential of matrix. Instead you can try to find right function in scipy library. Probably expm() function can be right for this perpuse.
I'm trying to define a complex custom likelihood function using pymc3. The likelihood function involves a lot of iteration, and therefore I'm trying to use theano's scan method to define iteration directly within theano. Here's a greatly simplified example that illustrates the challenge that I'm facing. The (fake) likelihood function I'm trying to define is simply the sum of two pymc3 random variables, p and theta. Of course, I could simply return p+theta, but the actual likelihood function I'm trying to write is more complicated, and I believe I need to use theano.scan since it involves a lot of iteration.
import pymc3 as pm
from pymc3 import Model, Uniform, DensityDist
import theano.tensor as T
import theano
import numpy as np
### theano test
theano.config.compute_test_value = 'raise'
X = np.asarray([[1.0,2.0,3.0],[1.0,2.0,3.0]])
### pymc3 implementation
with Model() as bg_model:
p = pm.Uniform('p', lower = 0, upper = 1)
theta = pm.Uniform('theta', lower = 0, upper = .2)
def logp(X):
f = p+theta
print("f",f)
get_ll = theano.function(name='get_ll',inputs = [p, theta], outputs = f)
print("p keys ",p.__dict__.keys())
print("theta keys ",theta.__dict__.keys())
print("p name ",p.name,"p.type ",p.type,"type(p)",type(p),"p.tag",p.tag)
result=get_ll(p, theta)
print("result",result)
return result
y = pm.DensityDist('y', logp, observed = X) # Nx4 y = f(f,x,tx,n | p, theta)
When I run this, I get the error:
TypeError: ('Bad input argument to theano function with name "get_ll" at index 0(0-based)', 'Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?')
I understand that the issue occurs in line
result=get_ll(p, theta)
because p and theta are of type pymc3.TransformedRV, and that the input to a theano function needs to be a scalar number of a simple numpy array. However, a pymc3 TransformedRV does not seem to have any obvious way of obtaining the current value of the random variable itself.
Is it possible to define a log likelihood function that involves the use of a theano function that takes as input a pymc3 random variable?
The problem is that your th.function get_ll is a compiled theano function, which takes as input numerical arrays. Instead, pymc3 is sending it a symbolic variable (theano tensor). That's why you're getting the error.
As to your solution, you're right in saying that just returning p+theta is the way to go. If you have scans and whatnot in your logp, then you would return the scan variable of interest; there is no need to compile a theano function here. For example, if you wanted to add 1 to each element of a vector (as an impractical toy example), you would do:
def logp(X):
the_sum, the_sum_upd = th.scan(lambda x: x+1, sequences=[X])
return the_sum
That being said, if you need gradients, you would need to calculate your the_sum variable in a theano Op and provide a grad() method along with it (you can see a toy example of that on the answer here). If you do not need gradients, you might be better off doing everything in python (or C, numba, cython, for performance) and using the as_op decorator.