I'm trying to convert the input tensor to a numpy array inside a custom keras loss function, after following the instructions here.
The above code runs on my machine with no errors. Now, I want to extract a numpy array with values from the input tensor. However, I get the following error:
"tensorflow.python.framework.errors_impl.InvalidArgumentError: You
must feed a value for placeholder tensor 'input_1' with dtype float
[[Node: input_1 = Placeholderdtype=DT_FLOAT, shape=[],
_device="/job:localhost/replica:0/task:0/cpu:0"]]"
I need to convert to a numpy array because I have other keras models that must operate on the input - I haven't shown those lines below in joint_loss, but even the code sample below doesn't run at all.
import numpy as np
from keras.models import Model, Sequential
from keras.layers import Dense, Activation, Input
import keras.backend as K
def joint_loss_wrapper(x):
def joint_loss(y_true, y_pred):
x_val = K.eval(x)
return y_true - y_pred
return joint_loss
input_tensor = Input(shape=(6,))
hidden1 = Dense(30, activation='relu')(input_tensor)
hidden2 = Dense(40, activation='sigmoid')(hidden1)
out = Dense(1, activation='sigmoid')(hidden2)
model = Model(input_tensor, out)
model.compile(loss=joint_loss_wrapper(input_tensor), optimizer='adam')
I figured it out!
What you want to do is use the Functional API for Keras.
Then your submodels outputs as tensors can be obtained as y_pred_submodel = submodel(x).
This is similar to how a Keras layer operates on a tensor.
Manipulate only tensors within the loss function. That should work fine.
Related
I am using keras model.predict to predict sentiments. I am using universal sentence embeddings. While predicting, I am getting the error described below.
Please provide your valuable insights.
Regards.
I have run the code for two sets of inputs. For say, input1, the prediction is obtained. While its not working for input 2.
Input 1 is the form : {(a1,[sents1]),....}
Input 2:{((a1,a2),[sents11])),...}
The input for predicting is the [sents1], [sents11] etc. extracted from this.
I could see the related question in (Keras model.predict function giving input shape error). But I don't know whether its resolved. Further, input1 is working.
import tensorflow as tf
import keras.backend as K
from keras import layers
from keras.models import Model
import numpy as np
def UniversalEmbedding(x):
return embed(tf.squeeze(tf.cast(x, tf.string)), signature="default", as_dict=True)["default"]
input_text = layers.Input(shape=(1,), dtype=tf.string)
embedding = layers.Lambda(UniversalEmbedding, output_shape=(embed_size,))(input_text)
dense = layers.Dense(256, activation='relu')(embedding)
pred = layers.Dense(category_counts, activation='softmax')(dense)
model = Model(inputs=[input_text], outputs=pred)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
sents1=list(input2.items())
with tf.Session() as session:
K.set_session(session)
session.run(tf.global_variables_initializer())
session.run(tf.tables_initializer())
# model.load_weights(.//)
for i,ch in enumerate(sents1):
new_text=ch[1]
if len(new_text)>1:
new_text = np.array(new_text, dtype=object)[:, np.newaxis]
predicts = model.predict(new_text, batch_size=32)
InvalidArgumentError: input must be a vector, got shape: [] [[{{node
lambda_2/module_1_apply_default/tokenize/StringSplit}} =
StringSplit[skip_empty=true,
_device="/job:localhost/replica:0/task:0/device:CPU:0"](lambda_2/module_1_apply_default/RegexReplace_1,
lambda_2/module_1_apply_default/tokenize/Const)]]
Try removing trailing blanks at the start of the sentence.
new_text.strip()
USE preprocessed sentences by splitting on space, creating some empty lists from trailing spaces, which cannot be embedded.
(Hope this answer is not too late)
Also could be some missing values in sentences, without text. Need to exclude these.
I am finding that my model has a tensor that is of shape (?,1,60). I want to know how I can reduce this to (?,60)? Not sure whether reshape or Flatten can be done with respect to a dimension. Any help?
Both layers will work, but in this case I prefer using keras.layers.Flatten. Here is an example:
from keras.layers import Input, Flatten
from keras.models import Model
import numpy as np
a = Input(shape=(1, 60))
b = Flatten()(a)
model = Model(inputs=a, outputs=b)
model.compile('sgd', 'mse')
pred = model.predict(x=np.ones(shape=(2, 1, 60)))
print(pred.shape)
I'm predicting image classes using Keras. It works in Google Cloud ML (GCML), but for efficiency need change it to pass base64 strings instead of json array. Related Documentation
I can easily run python code to decode a base64 string into json array, but when using GCML I don't have the opportunity to run a preprocessing step (unless maybe use a Lambda layer in Keras, but I don't think that is the correct approach).
Another answer suggested adding tf.placeholder with type of tf.string, which makes sense, but how to incorporate that into the Keras model?
Here is complete code for training the model and saving the exported model for GCML...
import os
import numpy as np
import tensorflow as tf
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.preprocessing import image
from tensorflow.python.platform import gfile
IMAGE_HEIGHT = 138
IMAGE_WIDTH = 106
NUM_CLASSES = 329
def preprocess(filename):
# decode the image file starting from the filename
# end up with pixel values that are in the -1, 1 range
image_contents = tf.read_file(filename)
image = tf.image.decode_png(image_contents, channels=1)
image = tf.image.convert_image_dtype(image, dtype=tf.float32) # 0-1
image = tf.expand_dims(image, 0) # resize_bilinear needs batches
image = tf.image.resize_bilinear(image, [IMAGE_HEIGHT, IMAGE_WIDTH], align_corners=False)
image = tf.subtract(image, 0.5)
image = tf.multiply(image, 2.0) # -1 to 1
image = tf.squeeze(image,[0])
return image
filelist = gfile.ListDirectory("images")
sess = tf.Session()
with sess.as_default():
x = np.array([np.array( preprocess(os.path.join("images", filename)).eval() ) for filename in filelist])
input_shape = (IMAGE_HEIGHT, IMAGE_WIDTH, 1) # 1, because preprocessing made grayscale
# in our case the labels come from part of the filename
y = np.array([int(filename[filename.index('_')+1:-4]) for filename in filelist])
# convert class labels to numbers
y = keras.utils.to_categorical(y, NUM_CLASSES)
########## TODO: something here? ##########
image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
# scores = build_model(decoded)
model = Sequential()
# model.add(decoded)
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
model.fit(
x,
y,
batch_size=64,
epochs=20,
verbose=1,
validation_split=0.2,
shuffle=False
)
predict_signature = tf.saved_model.signature_def_utils.build_signature_def(
inputs={'input_bytes':tf.saved_model.utils.build_tensor_info(model.input)},
########## TODO: something here? ##########
# inputs={'input': image }, # input name must have "_bytes" suffix to use base64.
outputs={'formId': tf.saved_model.utils.build_tensor_info(model.output)},
method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
)
builder = tf.saved_model.builder.SavedModelBuilder("exported_model")
builder.add_meta_graph_and_variables(
sess=K.get_session(),
tags=[tf.saved_model.tag_constants.SERVING],
signature_def_map={
tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: predict_signature
},
legacy_init_op=tf.group(tf.tables_initializer(), name='legacy_init_op')
)
builder.save()
This is related to my previous question.
Update:
The heart of the question is how to incorporate the placeholder that calls decode into the Keras model. In other words, after creating the placeholder that decodes the base64 string to a tensor, how to incorporate that into what Keras runs? I assume it needs to be a layer.
image = K.placeholder(shape=(), dtype=tf.string)
decoded = tf.image.decode_jpeg(image, channels=3)
model = Sequential()
# Something like this, but this fails because it is a tensor, not a Keras layer. Possibly this is where a Lambda layer comes in?
model.add(decoded)
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', input_shape=input_shape))
...
Update 2:
Trying to use a lambda layer to accomplish this...
import keras
from keras.models import Sequential
from keras.layers import Lambda
from keras import backend as K
import tensorflow as tf
image = K.placeholder(shape=(), dtype=tf.string)
model = Sequential()
model.add(Lambda(lambda image: tf.image.decode_jpeg(image, channels=3), input_shape=() ))
Gives the error: TypeError: Input 'contents' of 'DecodeJpeg' Op has type float32 that does not match expected type of string.
first of all I use tf.keras but this should not be a big problem.
So here is an example of how you can read a base64 decoded jpeg:
def preprocess_and_decode(img_str, new_shape=[299,299]):
img = tf.io.decode_base64(img_str)
img = tf.image.decode_jpeg(img, channels=3)
img = tf.image.resize_images(img, new_shape, method=tf.image.ResizeMethod.BILINEAR, align_corners=False)
# if you need to squeeze your input range to [0,1] or [-1,1] do it here
return img
InputLayer = Input(shape = (1,),dtype="string")
OutputLayer = Lambda(lambda img : tf.map_fn(lambda im : preprocess_and_decode(im[0]), img, dtype="float32"))(InputLayer)
base64_model = tf.keras.Model(InputLayer,OutputLayer)
The code above creates a model that takes a jpeg of any size, resizes it to 299x299 and returns as 299x299x3 tensor. This model can be exported directly to saved_model and used for Cloud ML Engine serving. It is a little bit stupid, since the only thing it does is the convertion of base64 to tensor.
If you need to redirect the output of this model to the input of an existing trained and compiled model (e.g inception_v3) you have to do the following:
base64_input = base64_model.input
final_output = inception_v3(base64_model.output)
new_model = tf.keras.Model(base64_input,final_output)
This new_model can be saved. It takes base64 jpeg and returns classes identified by the inception_v3 part.
Another answer suggested adding tf.placeholder with type of tf.string, which makes sense, but how to incorporate that into the Keras model?
In Keras you can access your selected Backend (in this case Tensorflow) by doing:
from keras import backend as K
This you already seem to import on your code. That will enable you to access some native methods and resources available on the backend of your choice. It is the case that Keras backend includes a method for creating placeholders, among other utilities. Regarding placeholders, we can see what the Keras docs indicates about them:
placeholder
keras.backend.placeholder(shape=None, ndim=None, dtype=None, sparse=False, name=None)
Instantiates a placeholder tensor and returns it.
It also gives some example on its use:
>>> from keras import backend as K
>>> input_ph = K.placeholder(shape=(2, 4, 5))
>>> input_ph._keras_shape
(2, 4, 5)
>>> input_ph
<tf.Tensor 'Placeholder_4:0' shape=(2, 4, 5) dtype=float32>
As you can see, this is returning a Tensorflow tensor, with shape (2,4,5) and of dtype float. If you had another backend while doing the example you would get another tensor object (a Theano one surely). You can therefore use this placeholder() to adapt the solution you got on your previous question.
In conclusion, you can use your backend imported as K (or whatever you want) to do calls on the methods and objects available on the backend of your choice, by doing K.foo.bar() on the desired method. I suggest you give a read to what the Keras Backend to explore more things that can be useful for you on future situations.
Update: As per your edit. Yes, this placeholder should be a layer in your model. Specifically, it should be the Input Layer of your model, as it holds your decoded image (as Keras needs it that way) to classify.
Passing the sample_weight parameter to GridSearchCV raises an error due to incorrect shape. My suspicion is that cross validation is not capable of handling the split of sample_weights accordingly with the dataset.
First part: Using sample_weight as a model parameter works beautifully
Let's consider a simple example, first without GridSearch:
import pandas as pd
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import GridSearchCV
import matplotlib.pyplot as plt
dataURL = 'https://raw.githubusercontent.com/mcasl/PAELLA/master/data/sinusoidal_data.csv'
x = pd.read_csv(dataURL, usecols=["x"]).x
y = pd.read_csv(dataURL, usecols=["y"]).y
occurrences = pd.read_csv(dataURL, usecols=["Occurrences"]).Occurrences
my_sample_weights = (1 - occurrences/10000)**3
my_sample_weights contains the importance that I assign to each observation in x, y, as the following picture shows. The points of the sinusoidal curve get higher weights than those forming the background noise.
plt.scatter(x, y, c=my_sample_weights>0.9, cmap="cool")
Let's train a neural network, first without using the information contained in my_sample_weights:
def make_model(number_of_hidden_neurons=1):
model = Sequential()
model.add(Dense(number_of_hidden_neurons, input_shape=(1,), activation='tanh'))
model.add(Dense(1, activation='linear'))
model.compile(optimizer='sgd', loss='mse')
return model
net_Not_using_sample_weight = make_model(number_of_hidden_neurons=6)
net_Not_using_sample_weight.fit(x,y, epochs=1000)
plt.scatter(x, y, )
plt.scatter(x, net_Not_using_sample_weight.predict(x), c="green")
As the following picture shows, the neural network tries to fit the shape of the sinusoidal but the background noise prevents it from a good fit.
Now, using the information of my_sample_weights , the quality of the prediction is a much better one.
Second part: Using sample_weight as a GridSearchCV parameter raises an error
my_Regressor = KerasRegressor(make_model)
validator = GridSearchCV(my_Regressor,
param_grid={'number_of_hidden_neurons': range(4, 5),
'epochs': [500],
},
fit_params={'sample_weight': [ my_sample_weights ]},
n_jobs=1,
)
validator.fit(x, y)
Trying to pass the sample_weights as a parameter gives the following error:
...
ValueError: Found a sample_weight array with shape (1000,) for an input with shape (666, 1). sample_weight cannot be broadcast.
It seems that the sample_weight vector has not been split in a similar manner to the input array.
For what is worth:
import sklearn
print(sklearn.__version__)
0.18.1
import keras
print(keras.__version__)
2.0.5
The problem is that as a standard, the GridSearch uses 3-fold cross-validation, unless explicity stated otherwise. This means that 2/3 data points of the data are used as training data and 1/3 for cross-validation, which does fit the error message. The input shape of 1000 of the fit_params doesn't match the number of training examples used for training (666). Adjust the size and the code will run.
my_sample_weights = np.random.uniform(size=666)
We developed PipeGraph, an extension to Scikit-Learn Pipeline that allows you to get intermediate data, build graph like workflows, and in particular, solve this problem (see the examples in the gallery at http://mcasl.github.io/PipeGraph )
I am trying to reuse the weight matrix from a previous layer. As a toy example I want to do something like this:
import numpy as np
from keras.layers import Dense, Input
from keras.layers import merge
from keras import backend as K
from keras.models import Model
inputs = Input(shape=(4,))
inputs2 = Input(shape=(4,))
dense_layer = Dense(10, input_shape=(4,))
dense1 = dense_layer(inputs)
def my_fun(my_inputs):
w = my_inputs[0]
x = my_inputs[1]
return K.dot(w, x)
merge1 = merge([dense_layer.W, inputs2], mode=my_fun)
The problem is that dense_layer.W is not a keras tensor. So I get the following error:
Exception: Output tensors to a Model must be Keras tensors. Found: dot.0
Any idea on how to convert dense_layer.W to a Keras tensor?
Thanks
It seems that you want to share weights between layers.
I think You can use denselayer as shared layer for inputs and inputs2.
merge1=dense_layer(inputs2)
Do check out shared layers # https://keras.io/getting-started/functional-api-guide/#shared-layers
I don't think that you can use the merge layer like this.
But to answer your question, you will probably have to create a custom layer which has tied weights. Look at this example.
Otherwise, the way to access the weights of a layer is to use get_weights() method on that layer, this will retrun a list of numpy arrays containing the weights. For the case of the Dense layer, it will contain weights and bias.
There are two cases for the solution, depending on what you are trying to do:
You would like to share the W matrix between your two operations, and the W matrix for these two operations are kept the same even if its value changed during training or for some other reason. Then you should use dense.weights[0] which is the W matrix as a tensor from your dense layer.
If you are only going to use the value of W matrix at the time of your code is written and this value is never going to change, then use K.constant(dense.get_weights[0]) which extracts the weights as numpy array and is converted into tensor.