I'm new to cntk and python. I have done machine learning in c# over the years with various libraries.
To help me learn cntk and python I doing a simple classification project with UCI wine data set. Right now I am only doing one hidden layer, I do plan to add more hidden layers after I get one hidden layer working.
I am having problems with creating a hidden Dense layer with dtype of numpy.float64. I know float32 is faster. I want to have my underlining data as float64 because of future projects that is a requirement.
I have searched the internet for help with no avail for answers. The last thing I have tried is creating the inputs with dtype=numpy.float64 and passing inputs to Dense, but that did not work either. In the past I have tried "with cntk.layers.default_options(dtype=numpy.float64)" inside a function but it does not accept numpy.float64 with or without a name. I have also tried calling update_signature(numpy.float64) on what is returned from Dense and I get the same error. I would tell you other things I have tried, but I honestly don't remember right now.
Here is my latest code:
inputsCount = 11
classesCount = 2
inputs = cntk.input_variable(shape=(inputsCount), dtype=numpy.float64)
model = cntk.layers.Dense(classesCount, activation=None)(inputs)
This is the error I get with it:
ValueError: Primitive op 'Times' passed operands 'Parameter('W', [],
[? x 2]), Placeholder('Placeholder135', [#], [11])' with different
DataTypes 'Float' and 'Double'.
Can you try this?
inputsCount = 11
classesCount = 2
with cntk.default_options(dtype=numpy.float64):
inputs = cntk.input_variable(shape=(inputsCount))
model = cntk.layers.Dense(classesCount, activation=None)(inputs)
Related
Today I suddenly started getting this error for no apparent reason, while I was running model.fit(). This used to work before, I am using TF 2.3.0, more specifically its Keras module.
The function is called on validation inside a generator, which is fed into model.predict().
Basically, I load a checkpoint, I resume training the network, and I make a prediction on validation.
The error keeps occurring even when training a model from scratch, and erasing all the related data. It's like if something has been hardcoded, somewhere, as I was able to run model.fit() up until a few hours ago.
I saw several solutions like THIS, but none of these variations really work for me, as they lead to more tricky error messages.
I even tried installing a different version of TF, thinking that this was due to some old version, but the error still occurs.
I will answer my own question, as this one was particularly tricky and none of the solutions I found on the internet has worked for me, probably because outdated.
I'll write down just the relevant part to add in the code, feel free to add more technical explanations.
I like using args for passing variables, but it can work without:
from tensorflow.python.keras.backend import set_session
from tensorflow.keras.models import load_model
import generator # custom generator
def main(args):
# open new session and define TF graph
args.sess = tf.compat.v1.Session()
args.graph = tf.compat.v1.get_default_graph()
set_session(args.sess)
# define training generator
train_generator = generator(args.train_data)
# load model
args.model = load_model(args.model_path)
args.model.fit(train_generator)
Then, in the model prediction function:
# In my specific case, the predict_output() function is
# called inside the generator function
def predict_output(args, x):
with args.graph.as_default():
set_session(args.sess)
y = model.predict(x)
return y
I want to make gaussian noise layer of Keras that is imposing noise with different stddev level to each column of dataset. However, since I am not know much about coding stuffs, there is a big problem that I cannot solve it by myself.
With source code of Keras gaussian noise layer,
I made a code like below :
def call(self, inputs, training=None):
def noised():
temp=inputs
for i in range(100):
temp[:,i]=temp[:,i]+K.random_normal(shape=
(len(inputs),1),mean=0.,stddev=self.stddev[i])
return temp
return K.in_train_phase(noised, inputs, training=training)
However, it shows an error like :
object of type 'Tensor' has no len()
I believe that the error comes from the different type of shape.
Because, the original code, which is like below :
def noised():
return inputs + K.random_normal(shape=K.shape(inputs),
mean=0.,
stddev=self.stddev)
is using symbolic type of shape(K.shape), and what I imposed is integer type of number(len()).
However, I have no idea the way to overcome the problem.
It would really be a great help for me if you give me some way to solve it.
Thank you so much for your assistance.
I know it's super late, but maybe it's still interesting for other people. I'm using Tensorflow 2.3.0 and I can just use the numpy slicing commands. So slice the tensor, apply the individual layers and merge them back together:
input = tf.keras.Input(shape=(None,3))
x1 = GaussianNoise(0.1)(input[:,:,0:1])
x2 = GaussianNoise(0.2)(input[:,:,1:2])
x3 = GaussianNoise(0.3)(input[:,:,2:3])
x = Concatenate()([x1,x2,x3])
I am using the class api to subclass a model based on keras.models.Model.
Is there some trick to getting the save_weights working?
Am seeing errors like
ValueError: Layer #0 (named "dense_S") expects 0 weight(s), but the saved weights have 2 element(s).
Am trying by_name=True and False.
EDIT: it seems that calling predict once, with ANY data, is needed to build the layers for some reason. It would be interesting to here a proper explanation from anyone who knows more.
I have a question regarding the implementation of a custom loss-function for my neural network.
I am currently trying to segment cells for a project and I decided to use a unet as it seems to work quite well. In order to improve my current model, I decided to follow the idea of the original paper of the unet (https://arxiv.org/abs/1505.04597) where they implemented a weight-map assigning thus more weight to pixels that are located in between cells that are tightly associated, as you can see in this picture: Example of a weight map.
I am currently using Keras for my unet and my problem is that I do not know how to give my weights to my model without creating any problem. My idea was to create a generator with the images and a 2-channeled array containing the labels in the first channel and the weights in the second channel, that way I can extract my weights and my labels easily in my custom loss function.
My code looks like that:
train_generator = zip(image_generator, label_generator, weight_generator)
for (img, label, weight) in train_generator:
img, label = adjustData(img, True, label)
label_weights = np.concatenate((label, weight),axis=3)
# This is the final generator
yield (img, label_weights)
As you can see, I construct the train_generator with three previously constructed generators, I adjust some things and then I yield my images and combined labels and weights.
Then, when I try to fit my model with fit_generator, I get this error: ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays.
I really do not know what to do and how to implement correctly what I want to do.
Thank you in advance for your answers.
This a part of the code for a Deconvolutional-Convoltional Generative Adversarial Network (DC-GAN)
discriminator.trainable = False
ganInput = Input(shape=(100,))
# getting the output of the generator
# and then feeding it to the discriminator
# new model = D(G(input))
x = generator(ganInput)
ganOutput = discriminator(x)
gan = Model(input=ganInput, output=ganOutput)
gan.compile(loss='binary_crossentropy', optimizer=Adam())
I do not understand what the line ganInput = Input(shape=(100,)) does.
Clearly ganInput is a variable but what is Input? Is it a function ?
If Input is a function then what will ganInput contain ?
Then it is ganInput is fed into the generator since it is an empty variable (assuming) it will not matter. Next ganOutput catches the output of the discriminator.
Then comes the problem. I read about the Model API but I do not understand fully what it does.
To summarise these are my problems : What is the role of ganInput and what is Input in the second line. And what is Model doing and what is it?
Using Keras with TensorFlow backend
COMPLETE SOURCE CODE : https://github.com/yashk2810/DCGAN-Keras/blob/master/DCGAN.ipynb
Please ask for any more clarification / details required. If you know the answer to even one of my queries I will request you to please answer it will be a huge help. Thanks
What is input: Notice the wildcard import of keras.layers. In context, Input is keras.layers.Input. Generally, if you see a function or class that wasn't defined or explicitly imported in Python, it got there via a wildcard import, i.e.
from keras.layers import *
That means import everything from keras.layers directly into the workspace.
What is model: The model object is essentially the interface for making neural networks with Keras.
You can read about model and keras.layers.Input at the model docs or at this model guide since I'm not very familiar with Keras.
What's going on in the example is they define generator and discriminator as Sequentials. But the GAN model is a little more complex than a standard old Sequential. The authors deal with that by marking the data that needs fed in at every iteration (in this case, just the random noise for the generator - ganInput) as a keras.layers.Input. Then, like you said, ganOutput catches the output of the discriminator. Since we have two distinct Sequentials that need wrapped together, the authors use the model API.