NumPy Data w/ TensorFlow "Not on the same graph" - python-3.x

Question

So i'm trying to build a model that takes in an image-like numpy file and builds a model based on the data. Each individual .npy file DOES fit in memory, however I need to assume that all loaded at the same time will not.
When I go to run it & do a test evaluation, it gives me this error:
ValueError: Tensor("IteratorV2:0", shape=(), dtype=resource) must be from the same graph as Tensor("MapDataset:0", shape=(), dtype=variant)
I'm not super experienced with TensorFlow and I'm trying to make everything as "best practice" as possible.
Here's my code:
train_fnames, train_labels, test_fnames, test_labels =\
spec_to_paths_and_labels('count_data/spec.csv')
train_fnames = 'count_data/' + train_fnames
test_fnames = 'count_data/' + test_fnames
def read_npy_file(item):
data = np.load(item.decode())
return data.astype(np.int32)
# gdsii_placeholder = tf.placeholder(tf.float32, shape=(None, 224, 224, 1))
# label_placeholder = tf.placeholder(tf.int32, shape=[1])
def cnn_model(features, labels, mode):
conv1 = tf.layers.conv2d(features, filters=32, kernel_size=(5, 5))
pool1 = tf.layers.max_pooling2d(conv1, pool_size=(2, 2), strides=2)
conv2 = tf.layers.conv2d(pool1, filters=64, kernel_size=(5, 5))
pool2 = tf.layers.max_pooling2d(conv2, pool_size=(2, 2), strides=2)
flat = tf.layers.flatten(pool2)
dense = tf.layers.dense(flat, units=1024, activation='relu')
dropout = tf.layers.dropout(inputs=dense, rate=0.4, training=True)
logits = tf.layers.dense(inputs=dropout, units=10)
predictions = {
"classes": tf.argmax(input=logits, axis=1),
"probabilities": tf.nn.softmax(logits, name="softmax_tensor")
}
loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
train_op = optimizer.minimize(
loss=loss,
global_step=tf.train.get_global_step())
return tf.estimator.EstimatorSpec(mode=tf.estimator.ModeKeys.TRAIN, loss=loss, train_op=train_op)
def main(_):
# load the dataset
data = tf.data.Dataset.from_tensor_slices(train_fnames)
data = data.map(lambda item: tuple(tf.py_func(
read_npy_file, [item], [tf.int32, ])))
gdsii_classifier = tf.estimator.Estimator(
model_fn=cnn_model, model_dir="/tmp/gdsii_classifier")
res = gdsii_classifier.evaluate(lambda: data)
print(res)
if __name__ == "__main__":
tf.app.run()

Answer 1

There are several little mistakes in your code:
First, you tried to use Estimator without wrapping your tf.data operations inside an input function. This causes an issue because tf.data operations create their own graph for data loading while Estimator expects to have all the control on the whole graph. When you then call estimator.evaluate(), it fails because the tensor is inside your previously built graph.
Secondly, you tried to call estimator.evaluate() without a label.
Then, after you use a tf.py_func in tf.data, you need to specify the shape of the different tensors as they can't be inferred by Tensorflow.
And finally, you need to batch your data as tf.layers expect ndims=4, and you need to pass the features as either float32 or float16.
You can easily patch these issues by doing something like this:
def input_fn(x, y):
def set_shape(tensor, shape):
tensor.set_shape(shape)
return tensor
data = tf.data.Dataset.from_tensor_slices((x, y))
data = data.map(lambda item, label: (tf.py_func(read_npy_file, [item], [tf.float32]), label))
data = data.map(lambda item, label: (set_shape(item, YOUR_SHAPE), label))
data = data.batch(1)
return data
...
estimator.evaluate(lambda: input_fn(x=test_fnames, y=test_labels))