I am using a DBN classification code. I have one output value which is an integer(value of the output integer can be anything 110,12 etc). So basically I am trying to predict something using the classification code by setting its no_of_outputs=1. But I think no_of_output=1 is messing up with the code by giving the error:
ValueError: y_i value out of bounds
Apply node that caused the error: CrossentropySoftmaxArgmax1HotWithBias(_dot22.0, b, Elemwise{Cast{int32}}.0)
Inputs shapes: [(10, 2), (2,), (10,)]
Inputs strides: [(16, 8), (8,), (4,)]
Inputs types: [TensorType(float64, matrix), TensorType(float64, vector), TensorType(int32, vector)]
Use the Theano flag 'exception_verbosity=high' for a debugprint of this apply node.
Kindly help me solve it.
That's index value out of bound error, you should check your codes to make sure:
Your index value is not NaN
If your index value is a valid integer, you should check whether it's out of bound actually
You can print your value by compiling a theano function, which returns the parameters you want to print.
Related
I use torch.normal to generate (1,n)-dimensional samples with an std vector that includes zero values. This doesn't generate any errors (when std[k] is zero the corresponding mean[k] is sampled).
However, torch.distributions.MultivariateNormal doesn't accept diagonal covariance matrix whose diagonal values are std[k]**2
I need the MultivariateNormal object to compute log_prob of the generated sample for which i used torch.normal
Code:
import torch
mean = torch.Tensor([1.00000,1.00000,1.00000,1.00000,1.00000,-1.00000,-1.00000,-1.00000,-1.00000])
std = torch.Tensor([17708.14062,68.16734,0.00000,0.00000,5917.79932,15390.00488, 0.00000,10070.79395,4994.94434])
x = torch.normal(mean, std)
torch.distributions.MultivariateNormal(loc=mean, covariance_matrix=torch.Tensor(np.diag(np.array(std)**2))).log_prob(x)
Error:
ValueError: Expected parameter covariance_matrix (Tensor of shape (9, 9)) of distribution MultivariateNormal(loc: torch.Size([9]), covariance_matrix: torch.Size([9, 9])) to satisfy the constraint PositiveDefinite(), but found invalid values
I don't get why torch.normal doesn't mind zero values in the std but normal.distributions.MultivariateNormal does.
I tried to look up special parameters that can be passed to the constructor to ignore this, like the 'allow_singular' parameter in the scipy library, but there seems to be none.
from scipy.stats import multivariate_normal
pi = multivariate_normal(mean=mean, cov=np.diag(np.array(std)**2), allow_singular=True).pdf(
np.array(action))
For instance, let's say I am working with a string tensor strs = ['12', '52', 'apple', '3'] (can be any dimension in general), and I want to map this to a float32 tensor using built-in tf operations. If I were to simply do tf.strings.to_number(strs), then there would be the following error message
StringToNumberOp could not correctly convert string: apple [Op:StringToNumber]
because clearly 'apple' cannot be interpreted as a number.
My question is: is there a way to specify some default float value (say -1) for the invalid entry to be converted to (in this case 'apple') with purely (vectorized) tensorflow operations. In this case, the desired output would be [12, 52, -1, 3].
The reason for doing this is because we want to perform all data preprocessing operations in a Keras layer and eliminate as much Python dependencies as possible.
I am using numpy and numpy.array2string to probe effects of small changes and numerical stability. I am baffled by the following exercise:
<Get vector data through prior code>
print(data.shape)
print(data.dtype)
variance = np.var(data)
print(variance.dtype)
print(variance)
print(np.array2string(variance, suppress_small=True,formatter={'float': '{: 8.55f}'.format}))
Which gives as output:
torch.Size([1, 64]) # shape of the data
torch.float32 # dtype of the data
float32 # dtype of the variance result
37166410.0
37166408.0000000000000000000000000000000000000000000000000000000
This makes me crosseyed: The array2string isn't just reformatting the value with a (patently absurd) number of digits, it seems to be changing the value itself well above the point where round off errors should matter. (It doesn't seem to be the formatter, as I can remove that and still see changed values.)
A float32 can easily represent either of those integer values directly.
What am I not understanding here? Please reaffirm my faith in objective reality.
UPDATE: No float32 cannot exactly represent 37166410.0, and I need reading glasses.
tensor(1.0000, grad_fn=<SumBackward0>)
tensor([1.])
Why are these two tensors not equivalent in pytorch?
I figured it out: when converting to numpy,
assert torch.sum(state_alphas).detach().numpy() == 1, torch.sum(state_alphas).detach().numpy()
AssertionError: 0.9999994
For some reason even though the Tensor is displayed as 1.000... it is not quite equal to 1.
I've been trying to implement a custom objective function in Keras (the negative log likelihood of the normal distribution)
Keras expects one argument for the ground truth tensor, and one for the predictions tensor; for y_pred,I'm passing a tensor that should represent a nx2 matrix where the first column is the mean of the distribution, and the second the precision.
My problem is that I haven't been able to get a clear idea how I properly slice y_pred before passing it into the likelihood function without yielding the error
'Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?'
While I understand that I'm feeding l_func arguments of the variable type when it expects an array,I don't seem to be able to grok how to properly split the input y_pred variable into its mean and precision components to plug into the likelihood function. Here are some attempts; if someone could enlighten me about how to proceed, I would greatly appreciate it.
def log_likelihood(y_true,y_pred):
mu = T.vector('mu')
beta = T.vector('beta')
x=T.vector('x')
likelihood = .5*(beta*(x-mu)**2)-T.log(beta/(2*np.pi))
l_func = function([mu,beta,x], likelihood)
return(l_func(y_pred[:,0],y_pred[:,1],y_true))
def log_likelihood(y_true,y_pred):
likelihood = .5*(y_pred[:,1]*(y_true-y_pred[:,0])**2)-T.log(y_pred[:,1]/(2*np.pi))
l_func = function([y_true,y_pred], likelihood)
return(l_func(y_true,y_pred))
def log_likelihood(y_true,y_pred):
mu=y_pred[:,0]
beta=y_pred[:,1]
x=y_true
mu_function=function([y_pred],mu)
beta_function=function([y_pred],beta)
id_function=function([y_true],x)
likelihood = .5*(beta_function(y_pred)*(id_function(y_true)-mu_function(y_pred))**2)-T.log(beta_function(y_pred)/(2*np.pi))
l_func = function([y_true,y_pred], likelihood)
return(l_func(y_true,y_pred))