I need to examine the output values of each layer of the network in theano-python.
I suppose : net.layers[i].output should do the job.
However, it seems net.layers[i].output is theano.tensor type. How to print it or save it in some file format can be read by human ?
Thanks.
You can use
net.layers[i].output.get_values()
to convert the tensor values into the Numpy array.
And subsequently, you use the following to store those values:
numpy.savetxt
Related
I hope to know how to build up the dataset with multiple NumPy arrays as a data channel.
I have multiple arrays for 4-channel data as input and 1-channel data as an output.
For instance,
Example of Data Structure - Tensor Form
I think this form of data can be built with the below function,
from torch.utils.data.dataset import Dataset
But I am still having some issues to assign the tensor properly, specify the input, and output.
I hope to get some ideas or guides for this problem.
Thank you so much!
Ok I'm working with keras, sequential.
I have a dataset, an array of 500 samples of 10 units each.
How do I feed it into the model ?
Am I supposed to flatten it to 1D and specify batch size as 10 ?
I have additional data associated with each sample. Which I don't need to feed into the model but need to keep track of as it is related to the sample.
The input to the model should specify the size of one sample, you don't need to flatten the input. In your case, the input size should be (10,). That is one sample has 10 values; the comma symbolizes a row is 1d.
The batch size can be anything you want/ what performs best for your dataset.
After training my dataset which has a number of categorical data using fastai's tabular model, I wish to read out the entity embedding and use it to map to my original data values.
I can see the embedding weights. The number of input don't seem to match anything, but maybe it is based on the unique categorical values in the train_ds.
To get that map, I would like to get the self.categories dictionary from the Categorify transform class. Is there anyway to get that from the data variable obtained by calling TabularList.from_df?
Or maybe someone can tell me a better way to get this map. I know the input df into the TabularList.from_df() is not it, because the number of rows are wrong. Most likely because df is splitted into train and valid subsets. But there is no easy way to obtain the train part of the TabularList to check just the train part.
It's strange I can't find any code example that shows this. Doesn't anyone else care to map the entity embedding value back to its original categorical value?
I found it.
It is in data.train_ds.inner_df.
i was trying to normalize my input and output data through the scikit-learn object MinMaxScaler. I have some questions about this process:
1 - How can i use the MinMaxScale inside a Lambda Layer to normalize both my input and output data in the range 0-1? Or i don't need to use the MinMaxScale at all for that purpose (like theres an easy way to do that using a Lambda Layer)?
2 - How can i do the inverse transformation of the output variables later? I've read someting about scaler.inverse_transform(), but not really sure where should i put it.
Thanks in advance!
To train a model using Keras, should I load all the images I have to an array to create something like
x_train, y_train
Or is there a better way to read the images on the fly while training. I am not looking for ImageDataGenerator class since my output is an array of points not classes based on directory names..
I managed to get my data csv file to contain the array of points and image file name in 9 columns as follows:
x1 x2 ..... x8 Image_file_name
You can use this data with ImageDataGenerator. You incorrectly assume that it needs folders for classes, but that only applies to flow_from_directory. The method flow_from_dataframe allows you to load data from a Pandas dataframe, from where you can load your data, for example:
idg = ImageDataGenerator(...)
df = pd.load_csv('your_data.csv')
generator = idf.flow_from_dataframe(directory='image folder', x_col = 'filename_column',
y_col = ['col1', 'col2', ..., 'coln'],
class_mode='other')
This generator will data from the dataframe, load the image filename in directory as specified by the value of x_col, and use the corresponding row to build the targets, which in this case will be a numpy array of the values of columns in y_col. More information about this method can be found in the keras documentation.
Loading the entire data set in memory in an array is not a great idea because the memory consumption could go out of control, so you should use a generator. ImageDataGenerator and flow_from_dataframe are a great way of loading images in Keras. Since you don't want to use ImageDataGenerator(can you mention why?) you can create your own generator function that loads chunks of images in memory. If you load your data in a generator make sure you use fit_generator and predict_generator functions.
To load unlabeled data you can do the following hack:
datagen = ImageDataGenerator()
test_data = datagen.flow_from_directory('.', classes=['directory_where_images_are_stored'])
For more information check out link [1].
[1] https://kylewbanks.com/blog/loading-unlabeled-images-with-imagedatagenerator-flowfromdirectory-keras