how to see the activation values of hidden layer in tensorflow - python-3.x

I implement stacked autoencoder using tensorflow library. it work properly. now i am trying to see hidden layer activation values (y1,y2,y3,y4,y5). but i do not find any way to see that. here is my code.
x= tf.placeholder(tf.float32,[None,784])
y_=tf.placeholder(tf.float32,[None,6])
k=190
l=180
m=150
n=130
o=100
num_of_epoch=10
w1=tf.Variable(tf.truncated_normal([784,k],stddev=0.1))
b1=tf.Variable(tf.zeros([k]))
w2=tf.Variable(tf.truncated_normal([k,l],stddev=0.1))
b2=tf.Variable(tf.zeros([l]))
w3=tf.Variable(tf.truncated_normal([l,m],stddev=0.1))
b3=tf.Variable(tf.zeros([m]))
w4=tf.Variable(tf.truncated_normal([m,n],stddev=0.1))
b4=tf.Variable(tf.zeros([n]))
w5=tf.Variable(tf.truncated_normal([n,o],stddev=0.1))
b5=tf.Variable(tf.zeros([o]))
w6=tf.Variable(tf.truncated_normal([o,6],stddev=0.1))
b6=tf.Variable(tf.zeros([6]))
y1=tf.nn.relu(tf.matmul(x,w1)+b1)
y2=tf.nn.relu(tf.matmul(y1,w2)+b2)
y3=tf.nn.relu(tf.matmul(y2,w3)+b3)
y4=tf.nn.relu(tf.matmul(y3,w4)+b4)
y5=tf.nn.relu(tf.matmul(y4,w5)+b5)
y=tf.nn.softmax(tf.matmul(y5,w6)+b6)
cross_entropy=tf.reduce_mean(-tf.reduce_sum(y_*tf.log(y),
reduction_indices=[1]))
train_step=tf.train.GradientDescentOptimizer(0.1).minimize(cross_entropy)
init=tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
for i in range(num_of_epoch):
train_data = {x:x_train,y_:y_train}
sess.run(train_step,feed_dict=train_data)
currect_prediction=tf.equal(tf.argmax(y,1),tf.argmax(y_,1))
accuracy=tf.reduce_mean(tf.cast(currect_prediction,tf.float32))
sess.run(accuracy,feed_dict={x:x_train,y_:y_train})
currect_prediction=tf.equal(tf.argmax(y,1),tf.argmax(y_,1))
accuracy=tf.reduce_mean(tf.cast(currect_prediction,tf.float32))
sess.run(accuracy,feed_dict= {x:x_test,y_:y_test})
if you know the way please share with me or give the any effective link where i can find the wright answer. thank in advance.

Related

what is model.training in pytorch?

hi i'm going through pytorch tutorial about transfer learning.
(https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html)
what is model.training for??
enter def visualize_model(model,num_images=6):
was_training=model.training
model.eval()
images_so_far=0
fig=plt.figure()
with torch.no_grad():
for i, (inputs,labels) in enumerate(dataloaders['val']):
inputs=inputs.to(device)
labels=labels.to(device)
outputs=model(inputs)
_,pred=torch.max(outputs,1)
for j in range(inputs.size()[0]):
images_so_far+=1
ax=plt.subplot(num_images//2,2,images_so_far)
ax.axis('off')
ax.set_title('predicted: {}'.format(class_names[preds[j]]))
imshow(inputs.cpu().data[j])
if images_so_far==num_images:
model.train(mode=was_training)
return
model.train(mode=was_training)code here
i cannot understand "model.train(model=was_training)". any help?? thank you so much
I think this will help (link)
All nn.Modules have an internal training attribute, which is changed by calling model.train() and model.eval() to switch the behavior of the model.
The was_training variable stores the current training state of the model, calls model.eval(), and resets the state at the end using model.train(training=was_training).
You can find great answers in pytorch discuss forum ;)
I wonder why they use model.train in the test session. why do they put that code inside the with torch.no_grad()? Isn't it obvious that was_training=false?
It is a bit misleading usage of train because train can be used to put the model in inference (evaluation) mode as well:
>>> model.train(mode=True)
>>> model.training
True # <- train mode
>>> model.train(mode=False)
False # <- eval mode
I agree it is not ideal, a more appropriate formulation would have been simply:
>>> model.eval()

confusion matrix for multiclass data

I have fit the following model to my 7 classes data and I want to create a confusion matrix for my model:
history1 = model1.fit(data_generator.flow(train_x, to_categorical(train_y),batch_size=BATCH_SIZE),
steps_per_epoch=len(train_x) / BATCH_SIZE,
validation_data=data_generator.flow(val_x, to_categorical(val_y),batch_size=BATCH_SIZE),
validation_steps=len(val_x) / BATCH_SIZE,epochs=NUM_EPOCHS)
Also when I did this, predicting the train set results:
y_train_pred = model1.predict(train_x)
cm_train = confusion_matrix(train_y, y_train_pred)
It gave me this error:
Classification metrics can't handle a mix of unknown and multiclass targets
Can you please guide me how to do it?
You seem to be using
sklearn.metrics.confusion_matrix
try using
from sklearn.metrics import multilabel_confusion_matrix
instead.

How to output the loss gradient backpropagation path through a PyTorch computational graph

I have implemented a new loss function in PyTorch.
#model_1 needs to be trained
outputs = model_1(input)
loss = myloss(outputs,labels)
#output is how much to resize an image
#label give the image file index
#Below I am explaining myloss() function
org_file_name = pic_ + str(labels[0]) + ".png"
new_image = compress(org_image,outputs)
accuracy_loss = run_pretrained_yolov3(org_image, new_image)
#next two lines are modifying the same DAG
prev_loss = torch.mean((outputs-labels)**2)
new_loss = (accuracy_loss/prev_loss.item())*prev_loss
new_loss.backward()
Can anyone plz help me suggesting how can I know regarding how the loss gradient backpropagation through the computational graph?
[i.e., Actually, inside the myloss() function, I used some other pre-trained model applied in testing mode to get the difference or final loss value.] Now I want to know whether my new_loss.grad backpropagated through model1 or first through yolov3 then through model1? pretrained yolov3 is used on testing mode only.
I have tried tensorboard, it's not providing me that option. Any suggestions will be highly helpful.

unable to use Trained Tensorflow model

I am new to Deep Learning and Tensorflow. I retrained a pretrained tensorflow inceptionv3 model as saved_model.pb to recognize different type of images but when I tried to use the fie with below code.
with tf.Session() as sess:
with tf.gfile.FastGFile("tensorflow/trained/saved_model.pb",'rb') as f:
graph_def = tf.GraphDef()
tf.Graph.as_graph_def()
graph_def.ParseFromString(f.read())
g_in=tf.import_graph_def(graph_def)
LOGDIR='/log'
train_writer=tf.summary.FileWriter(LOGDIR)
train_writer.add_graph(sess.graph)
it gives me this error -
File "testing.py", line 7, in <module>
graph_def.ParseFromString(f.read())
google.protobuf.message.DecodeError: Error parsing message
I tried many solution I can find for this problem and modules in tensorflow/python/tools which uses the graph_def.ParseFromString(f.read()) function are giving me same error. Please tell me how to solve this or tell me the way in which I can avoid ParseFromString(f.read()) function. Any help would be appreciated. Thank you!
Please use the frozen_inference_graph.pb to load the model,
than to use the saved_model.pb
Model_output
- saved_model
- saved_model.pb
- checkpoint
- frozen_inference_graph.pb # Main model
- model.ckpt.data-00000-of-00001
- model.ckpt.index
- model.ckpt.meta
- pipeline.config
I am assuming that you saved your trained model using tf.saved_model.Builder provided by TensorFlow, in which case you could possibly do something like:
Load model
export_path = './path/to/saved_model.pb'
# We start a session using a temporary fresh Graph
with tf.Session(graph=tf.Graph()) as sess:
'''
You can provide 'tags' when saving a model,
in my case I provided, 'serve' tag
'''
tf.saved_model.loader.load(sess, ['serve'], export_path)
graph = tf.get_default_graph()
# print your graph's ops, if needed
print(graph.get_operations())
'''
In my case, I named my input and output tensors as
input:0 and output:0 respectively
'''
y_pred = sess.run('output:0', feed_dict={'input:0': X_test})
To give some more context here, this is how I saved my model which can be loaded as above.
Save model
x = tf.get_default_graph().get_tensor_by_name('input:0')
y = tf.get_default_graph().get_tensor_by_name('output:0')
export_path = './models/'
builder = tf.saved_model.builder.SavedModelBuilder(export_path)
signature = tf.saved_model.predict_signature_def(
inputs={'input': x}, outputs={'output': y}
)
# using custom tag instead of: tags=[tf.saved_model.tag_constants.SERVING]
builder.add_meta_graph_and_variables(sess=obj.sess,
tags=['serve'],
signature_def_map={'predict': signature})
builder.save()
This will save your protobuf ('saved_model.pb') in the said folder ('models' here) which can then be loaded as stated above.
Have you passed as_text=False when saving a model? Please have a look at: TF save/restore graph fails at tf.GraphDef.ParseFromString()

How to use cross_val_predict to predict probabilities for a new dataset?

I am using sklearn's cross_val_predict for training like so:
myprobs_train = cross_val_predict(LogisticRegression(),X = x_old, y=y_old, method='predict_proba', cv=10)
I am happy with the returned probabilities, and would like now to score up a brand-new dataset. I tried:
myprobs_test = cross_val_predict(LogisticRegression(), X =x_new, y= None, method='predict_proba',cv=10)
but this did not work, it's complaining about y having zero shape. Does it mean there's no way to apply the trained and cross-validated model from cross_val_predict on new data? Or am I just using it wrong?
Thank you!
You are looking at a wrong method. Cross validation methods do not return a trained model; they return values that evaluate the performance of a model (logistic regression in your case). Your goal is to fit some data and then generate prediction for new data. The relevant methods are fit and predict of the LogisticRegression class. Here is the basic structure:
logreg = linear_model.LogisticRegression()
logreg.fit(x_old, y_old)
predictions = logreg.predict(x_new)
I have the same concern as #user3490622. If we can only use cross_val_predict on training and testing sets, why y (target) is None as the default value? (sklearn page)
To partially achieve the desired results of multiple predicted probability, one could use the fit then predict approach repeatedly to mimic the cross-validation.

Resources