I am using the keras R package in RStudio.
I want to fit a model that uses a customized loss function; specifically, (-1)* log-likelihood for a Poisson model. I am smoothing the logarithm as ln(0.0001 + x^2)/2. Following the example in this article I write
K <- backend()
poisson <- function(y_true, y_pred){
K$sum(y_pred - y_true * K$log(y_pred^2 + 1e-4)/2 + lgamma(y_true+1))
}
Here I am mixing functions used by Keras from the "backend" source, such as K$log, and R function, such as lgamma, which I used because K$lgamma threw an error.
The rest of the commands in the example did run and produced some output.
Questions:
1) Can one mix and match functions in keras running in RStudio?
2) How can I test what the function is doing? I tried
poisson(1:5,3:7)
And I get the error
Error in py_call_impl(callable, dots$args, dots$keywords) :
TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.
I guess it’s a bad idea mix both R and Keras function in custom loss function and here why: y_true, y_pred it’s not R vectors, but Keras tensors and this is source of your error.
Related
I'm writing a custom loss function for a (sort of) semantic segmentation task where I compute binary crossentropy (using keras backend) for the target (2-d array) and predictions. The overall loss is a sum of 4 different loss functions.
In two of those functions I need to make a customized target array and compute binary crossentropy with these targets and the predictions. Here I would like to ignore and not calculate the loss where label is 1 (foreground) in the target array.
In pytorch's nll_loss() there is a parameter 'ignore_index' which is similar to what I'm trying to do.
I'm trying to achieve something like:
def binary_crossentropy(y_true, y_pred, ignore_label=1):
if ignore_label == 1:
return -(1-y_true)*log(1-y_pred)
if ignore_label == 0:
return -y_true*log(y_pred)
return -(y_true*log(y_pred) + (1-y_true)*log(1-y_pred))
But for a keras custom loss function.
I have a pytorch tensor k x (n+k-1) tensor w with requires_grad=True. I want to transform it into a kxn tensor p also with as such: p[i] = w[i][i:i+n]. How do I do this, such that by calling backward() on a loss function of p in the end, I will learn w?
Any sort of indexing operation would do, with the backward function being <CopySlices>
A naive way of doing this would be using simple python indexing:
w_unrolled = torch.zeros(p.size())
for i in range(w.shape[0]):
w_unrolled[i] = w[i][i:i+n]
loss = criterion(w_unrolled, p)
You can then reduce your loss via mean/sum on whichever axis. Note that while this will work, it is inefficient; the optimal way would be to use a native indexing function to speed things up.
I am doing one project by using Keras with tensorflow back-end. For business reason, I need to give high weight to some kind of errors, so I implement one customized loss function. For example:
error = np.abs(y_true - predict)
if error > low_limit:
error = error * 10
I found that this customized loss function really changed the error value displayed during the training. What I am wondering is whether this new loss function really can change the behavior of backpropagation during the training? Because I did not see too much difference from the weights of my model.
Short answer: yes, this loss function does change the behavior of BP, but I quite surprise that this loss function works...
Anyway, I feel the following loss function makes more sense
def my_mae( y_true, y_pred, low_bound=1e-3, coef=10. ) :
raw_mae = keras.losses.mae( y_true, y_pred )
mask = K.cast( raw_mae <= low_bound, dtype='float32' )
#new_mae = mask * raw_mae * coef + (1-mask) * raw_mae
new_mae = (1 + mask * (coef-1)) *raw_mae
return new_mae
which gives extra loss when a sample loss is too low.
I'm new to Pytorch and neural network programming but I've an issue I encountered and I'm not able to solve it on my own. My data are numpy arrays of 1 and 0. But when I try to train my net, I get this error :
RuntimeError: Expected object of type torch.ByteTensor but found type torch.FloatTensor for argument #2 'mat2'
the line where the error comes from is in the forward method of my net
x = self.fc1(x)
I've tried these to convert my tensors but I still get the error :
x = x.type('torch.ByteTensor')
and
x.byte()
x.byte() returns what you need, but it's not an "inplace" method. Try doing:
x = x.byte()
I've been trying to implement a custom objective function in Keras (the negative log likelihood of the normal distribution)
Keras expects one argument for the ground truth tensor, and one for the predictions tensor; for y_pred,I'm passing a tensor that should represent a nx2 matrix where the first column is the mean of the distribution, and the second the precision.
My problem is that I haven't been able to get a clear idea how I properly slice y_pred before passing it into the likelihood function without yielding the error
'Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?'
While I understand that I'm feeding l_func arguments of the variable type when it expects an array,I don't seem to be able to grok how to properly split the input y_pred variable into its mean and precision components to plug into the likelihood function. Here are some attempts; if someone could enlighten me about how to proceed, I would greatly appreciate it.
def log_likelihood(y_true,y_pred):
mu = T.vector('mu')
beta = T.vector('beta')
x=T.vector('x')
likelihood = .5*(beta*(x-mu)**2)-T.log(beta/(2*np.pi))
l_func = function([mu,beta,x], likelihood)
return(l_func(y_pred[:,0],y_pred[:,1],y_true))
def log_likelihood(y_true,y_pred):
likelihood = .5*(y_pred[:,1]*(y_true-y_pred[:,0])**2)-T.log(y_pred[:,1]/(2*np.pi))
l_func = function([y_true,y_pred], likelihood)
return(l_func(y_true,y_pred))
def log_likelihood(y_true,y_pred):
mu=y_pred[:,0]
beta=y_pred[:,1]
x=y_true
mu_function=function([y_pred],mu)
beta_function=function([y_pred],beta)
id_function=function([y_true],x)
likelihood = .5*(beta_function(y_pred)*(id_function(y_true)-mu_function(y_pred))**2)-T.log(beta_function(y_pred)/(2*np.pi))
l_func = function([y_true,y_pred], likelihood)
return(l_func(y_true,y_pred))