ValueError: too many dimensions 'str' python in pytorch - pytorch

I keep getting the error:
ValueError: too many dimensions 'str'
I attach my colab notebook to have a look at it. Have not found anything online yet that helps me solve the problem.
link:
https://colab.research.google.com/drive/1ikol2D8mmiIPKhNHbcFlTfVpuU_Gf9BZ?usp=sharing

I've seen this error as well in my Jupyter notebook. I can reproduce it with the following simple code:
Input:
tensor(['a'])
Output:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-55-3bdb0dbfafc2> in <module>
----> 1 tensor(['a'])
...(stacktrace)...
ValueError: too many dimensions 'str'
Apparently PyTorch tensors are different from Numpy arrays in that they don't work with strings, only integers, floats, and booleans.
The above error indicates that there are too many strings passed to the tensor (i.e. even one string is too many). When I change the code to the following, it works fine:
Input:
tensor([1])
Output:
tensor([1])

I haven't checked your notebook yet, but I just solved mine with the same error. Just double check if all training datasets and labels are converted to numeric values or tensors. If you have multiple columns in your dataframe, remove the one that does not need to feed into the training loop.

Related

Passing a python list to keras model.fit

So right now I'm using keras and training my model works perfectly fine, but I have to pass my data as numpy ndarray. So I have to convert my list of data to numpy ndarray first and then pass it to keras for training. When I try to pass my python list/array, even tho it's the same shape as numpy array I get back errors. Is there any way to not use numpy for this or am I stuck with it?
Can you further explain your problem. What is the error message you are getting and are you getting this error during training or predicting?
Also if you could post some code samples that would help to

scipy.convolve gives "ValueError: object too deep for desired array" with 3D array and 3D kernel

I am using Python 3 on Anacona Spyder on CentOS 7.
The following call
scipy.convolve(nda, box)
gives the following error message.
ValueError: object too deep for desired array
nda and box have the same type and dimensions.
np.shape(nda)
Out[51]: (70, 70, 70)
np.shape(box)
Out[52]: (3, 3, 3)
type(nda)
Out[53]: numpy.ndarray
type(box)
Out[54]: numpy.ndarray
It is my understanding that scipy.convolve can handles multidimensional objects. I cannot understand this error message.
The name scipy.convolve is an alias for numpy.convolve, and the NumPy version accepts only one-dimensional input. (This potential confusion is one of the reasons SciPy is deprecating the use of the NumPy names in the scipy namespace.)
You probably want scipy.ndimage.convolve or scipy.signal.convolve. (Why SciPy has independent implementations of convolve in two subpackages is a whole 'nother topic.)

UnboundLocalError: local variable 'photoshop' referenced before assignment

I am working on Dog Breed classifier, I am getting the following the error when I run the train code for my model.
I tried to downgrade the pillow version, but still facing the same issue.
Error shown in the line model_scratch:
---------------------------------------------------------------------------
UnboundLocalError Traceback (most recent call last)
<ipython-input-12-360fef19693f> in <module>
1 #training the model
2 model_scratch = train(5, loaders_scratch, model_scratch, optimizer_scratch,
----> 3 criterion_scratch)
<ipython-input-11-c90fddb93f0d> in train(n_epochs, loaders, model, optimizer, criterion)
9 #train model
10 model.train()
---> 11 for batch_idx, (data,target) in enumerate(loaders['train']):
12
13 # zero the parameter (weight) gradients
UnboundLocalError: local variable 'photoshop' referenced before assignment
This is a known issue in Pillow 6.0.0. Based on the line number information in your linked full stack trace, I think your downgrade didn't succeed, and you are still using 6.0.0. Either downgrading to 5.4.1, or building from the latest source, should fix this problem, although the latter option is probably a little difficult for the average user.

Using Spark DataFrame directly in Keras (databricks)

I have some text that I am looking to classify with keras. I have created a pipeline that takes the text and does some transformations on it and eventually one hot encodes it.
Now, I want to pass that OneHotEncoded column directly into keras in databricks along with the label column, but I cannot seem to do it. All of the examples that I see seem to start with a pandas dataframe and then convert to to a numpy array. But it seems counterproductive to take my pyspark dataframe and convert it.
model.fit(trainingData.select('featuresFirst'), trainingData.select('label'))
gives me:
AttributeError: 'DataFrame' object has no attribute 'values'
model.fit(trainingData.select('featuresFirst').collect(), trainingData.select('label').collect())
gives me:
AttributeError: ndim
What am I missing here?

Gensim: KeyedVectors.train()

I downloaded Wikipedia word vectors from here. I loaded the vectors with:
model_160 = KeyedVectors.load_word2vec_format(wiki_160_path, binary=False)
and then want to train them with:
model_160.train()
I get the error back:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-11-22a9f6312119> in <module>()
----> 1 model.train()
AttributeError: 'KeyedVectors' object has no attribute 'train'
My question is now:
It seems like KeyedVectors has no train function, but I want to continue training the vectors on my personal sentences, instead of just using the Wikipedia vectors. How is this possible?
Thanks in advance, Jan
You can't use KeyedVectors for that.
From the documentation:
Word vector storage and similarity look-ups.
The word vectors are considered read-only in this class.
And also:
The word vectors can also be instantiated from an existing file on disk in the word2vec C format as a KeyedVectors instance.
[...]
NOTE: It is impossible to continue training the vectors loaded from the C format
because hidden weights, vocabulary frequency and the binary tree is
missing.

Resources