I want to construct a multivariate Normal model in PyMC3 in which the mean value and precision matrix involve probabilistic variables. h is meant to act as a latent variable in an larger project to which this code snippet belongs.
When I run the code provided below, I get the error message shown, and I'm not sure exactly how to interpret it. As far as I can see, the dimension of the mean value of the MvNormal (2-row column vector) match the dimension of the precision matrix B (2 x 2 matrix), so I don't expect it's the dimensions of these objects that are causing the problem. I don't know what other variables could be causing some error related to dimensions to be thrown up though. Can anyone shed some light on this please?
Here is the code:
import pymc3 as pm
import theano.tensor as tt
with pm.Model() as model:
# A matrix
a1 = pm.Uniform('a1', 0., 1.)
a2 = pm.Uniform('a2', 0., 1.)
ix = ([0, 0, 1, 1], [0, 1, 0, 1])
A = tt.eye(2)
A = tt.set_subtensor(A[ix], [a1, a2, 1, 0])
# B matrix
b1 = pm.Uniform('b1', 0., 1.)
b2 = pm.Uniform('b2', 0., 1.)
ix = ([0, 1], [0, 1])
B = tt.eye(2)
B = tt.set_subtensor(B[ix], [b1 ** 2, b2 ** 2])
# Model
y0 = pm.Normal('y0', mu=0., sd=1., observed=0)
y1 = pm.Normal('y1', mu=1., sd=1., observed=1)
s_v = tt.stack([y1, y0]).T
h = pm.MvNormal("h", mu=pm.math.dot(A, s_v), tau=B)
Error message:
h = pm.MvNormal("h", mu=pm.math.dot(A, s_v), tau=B)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/distributions/distribution.py", line 42, in __new__
return model.Var(name, dist, data, total_size)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/model.py", line 809, in Var
total_size=total_size, model=self)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/model.py", line 1209, in __init__
self.logp_elemwiset = distribution.logp(self)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/distributions/multivariate.py", line 274, in logp
quaddist, logdet, ok = self._quaddist(value)
File "/Users/Joel/PycharmProjects/AR(2)/venv/lib/python3.6/site-packages/pymc3/distributions/multivariate.py", line 85, in _quaddist
raise ValueError('Invalid dimension for value: %s' % value.ndim)
ValueError: Invalid dimension for value: 0```
I believe that you are missing the "shape" argument in the pm.MvNormal call, which lets it handle the right size of values. For example, if you have 7 variables, set shape=7.
Related
I'm trying to make a neural network (using YT guide, but I had to change data input code) and I need the batched dataset for the train function to work properly (idk why, not event sure on it).
But when I try to convert a train data list to Dataset using tensorflow.data.Dataset.from_tensor_slices(train_data)) I receive a error message:
InvalidArgumentError
{{function_node __wrapped__Pack_N_3_device_/job:localhost/replica:0/task:0/device:GPU:0}} Shapes of all inputs must match: values[0].shape = [105,105,3] != values[2].shape = [1] [Op:Pack] name: 0
The train_data list consists of 560 lists, each with 3 elements inside:
<tf.Tensor: shape=(105, 105, 3), dtype=float32, numpy = array([[["105x105 3-dimensional image with my face"]]]. dtype=float32)>
<tf.Tensor: shape=(105, 105, 3), dtype=float32, numpy = array([[["different image with the same properties"]]] dtype=float32)>
<tf.Tensor: shape=(1,), dtype=float32, numpy=array(["1. or 0. (float), a label, showing if these pictures are actually the pictures of the same person"], dtype=float32)>
I am pretty sure that all of the shapes in the train_data list are exactly as described.
Some data about shapes using .shape method
train_data.shape #"AttributeError: 'list' object has no attribute 'shape'" - main list
train_data[0].shape #"AttributeError: 'list' object has no attribute 'shape'" - sublist, with 3 elements
train_data[0][0].shape #"TensorShape([105, 105, 3])" - first image
train_data[0][0][0].shape #"TensorShape([105, 3])" - first row of image pixels, ig
train_data[0][0][0][0].shape #"TensorShape([3])" - pixel in the left upper corner
That's what I tried to do:
The label of the image pairs (1. or 0.) was previosly just an integer. Then, I received an error saying that everything here should be the same type of float32. Then, I tried to convert it to tensor, but it changed nothing except the last part of the current error message, it used to say "values[2].shape = []" before.
I really have no idea what could lead to the error. I don't have any Tensorflow usage experience.
sorry if my engrish is bad
Edit: here is the code that takes the images out of certain directory.
May cause eye bleeding
for i in os.listdir("t"):
for ii in os.listdir(os.path.join("t", i)):
td.append([
[
tensorflow.expand_dims(
tensorflow.io.decode_jpeg(
tensorflow.io.read_file(os.path.join("t", i, ii) + "\\" + os.listdir(os.path.join("t", i, ii))[0])) / 255, 0),
tensorflow.expand_dims(
tensorflow.io.decode_jpeg(
tensorflow.io.read_file(os.path.join("t", i, ii) + "\\2.jpeg")) / 255, 0)],
tensorflow.convert_to_tensor(
float(
os.listdir(os.path.join("t", i, ii))[0][0]
)
)
])
I added some spaces in order to make it a bit more readable. td = train_data.
Yea, I could've messed something up there.
Edit 2: Answering Mohammad's question, there is the output data shape of the code they gave me:
td.shape #AttributeError: 'list' object has no attribute 'shape' - main list
td[0].shape #AttributeError: 'list' object has no attribute 'shape' - sublist, with a list and a label
td[0][0].shape #AttributeError: 'list' object has no attribute 'shape' - subsublist, with 2 images
td[0][1].shape #TensorShape([]) - label
td[0][0][0].shape #TensorShape([1, 105, 105, 3]) - first image
td[0][0][1].shape #TensorShape([1, 105, 105, 3]) - second image
It can be shown as:
train_data = [ [[x1, x2], y], [[x1, x2], y], ... ]
Replicating the problem:
x1 = tf.random.normal((105,105,3))
x2 = tf.random.normal((105,105,3))
y = tf.random.normal((1,))
array_list = [[x1, x2, y]] * 560
tf.data.Dataset.from_tensor_slices(array_list)
#InvalidArgumentError ... values[0].shape = [105,105,3] != values[2].shape = [1]
Fix:
#flatten to a single list
flatten_list = sum(array_list, [])
#Separate features and labels
X = tf.squeeze(tf.stack(flatten_list[::3]))
y = tf.squeeze(tf.stack(flatten_list[2::3]))
#construct dataset iterator
ds = tf.data.Dataset.from_tensor_slices((X, y))
for data in ds.take(1):
print(data)
Your data is in this shape right now...
x1 = tf.random.normal((105, 105, 3))
x2 = tf.random.normal((105, 105, 3))
y = tf.random.normal((1,))
train_list = [[[x1,x2] , y] , [[x1,x2] , y] , [[x1,x2] , y] , [[x1,x2] , y]]
x1 = [train_list[x][:1][0][0] for x in range(len(train_list))]
x2 = [train_list[x][:1][0][1] for x in range(len(train_list))]
y = [train_list[x][1:] for x in range(len(train_list))]
tf.data.Dataset.from_tensor_slices(((x1 , x2) , y))
<TensorSliceDataset element_spec=((TensorSpec(shape=(105, 105, 3), dtype=tf.float32, name=None), TensorSpec(shape=(105, 105, 3), dtype=tf.float32, name=None)), TensorSpec(shape=(1, 1), dtype=tf.float32, name=None))>
Or Change the Code when you are Loading Images and Labels from Disks This will save time
x1 = []
x2 = []
y = []
for i in os.listdir("t"):
for ii in os.listdir(os.path.join("t", i)):
x1.append(
tensorflow.expand_dims(
tensorflow.io.decode_jpeg(
tensorflow.io.read_file(os.path.join("t", i, ii) + "\\" + os.listdir(os.path.join("t", i, ii))[0])) / 255, 0))
x2.append(tensorflow.expand_dims(
tensorflow.io.decode_jpeg(
tensorflow.io.read_file(os.path.join("t", i, ii) + "\\2.jpeg")) / 255, 0)
)
y.append(tensorflow.convert_to_tensor(
float(
os.listdir(os.path.join("t", i, ii))[0][0]
)
))
tf.data.Dataset.from_tensor_slices(((x1 , x2) , y))
Given the following tensors x and y with shapes [3,2,3] and [3,2]. I want to multiply the tensors along the 2nd dimension, this is expected to be a kind of dot product and scaling along the axis and return a [3,2,3] tensor.
import torch
a = [[[0.2,0.3,0.5],[-0.5,0.02,1.0]],[[0.01,0.13,0.06],[0.35,0.12,0.0]], [[1.0,-0.3,1.0],[1.0,0.02, 0.03]] ]
b = [[1,2],[1,3],[0,2]]
x = torch.FloatTensor(a) # shape [3,2,3]
y = torch.FloatTensor(b) # shape [3,2]
The expected output :
Expected output shape should be [3,2,3]
#output = [[[0.2,0.3,0.5],[-1.0,0.04,2.0]],[[0.01,0.13,0.06],[1.05,0.36,0.0]], [[0.0,0.0,0.0],[2.0,0.04, 0.06]] ]
I have tried the two below but none of them is giving the desired output and output shape.
torch.matmul(x,y)
torch.matmul(x,y.unsqueeze(1).shape)
What is the best way to fix this?
This is just broadcasted multiply. So you can insert a unitary dimension on the end of y to make it a [3,2,1] tensor and then multiply by x. There are multiple ways to insert unitary dimensions.
# all equivalent
x * y.unsqueeze(2)
x * y[..., None]
x * y[:, :, None]
x * y.reshape(3, 2, 1)
You could also use torch.einsum.
torch.einsum('abc,ab->abc', x, y)
I have a tensor T with dimension (d1 x d2 x d3 x ... dk) and a tensor I with dimension (p x q). Here, I contains coordinates of T but q < k, each column of I corresponds to a dimension of T. I have another tensor V of dimension p x di x ...dj where sum([di, ..., dj]) = k - q. (di, .., dj) corresponds to missing dimensions from I. I need to perform T[I] = V
A specific example of such problem using numpy array posted here[1].
The solution[2] uses fancy indexing[3] which relies on numpy.index_exp. In case of pytorch such option is not available. Is there any alternative way to mimic this in pytorch without using loops or casting tensors to numpy array?
Below is a demo:
import torch
t = torch.randn((32, 16, 60, 64)) # tensor
i0 = torch.randint(0, 32, (10, 1)).to(dtype=torch.long) # indexes for dim=0
i2 = torch.randint(0, 60, (10, 1)).to(dtype=torch.long) # indexes for dim=2
i = torch.cat((i0, i2), 1) # indexes
v = torch.randn((10, 16, 64)) # to be assigned
# t[i0, :, i2, :] = v ?? Obviously this does not work
[1] Slice numpy array using list of coordinates
[2] https://stackoverflow.com/a/42538465/6422069
[3] https://numpy.org/doc/stable/reference/generated/numpy.s_.html
After some discussion in the comments, we arrived at the following solution:
import torch
t = torch.randn((32, 16, 60, 64)) # tensor
# indices
i0 = torch.randint(0, 32, (10,)).to(dtype=torch.long) # indexes for dim=0
i2 = torch.randint(0, 60, (10,)).to(dtype=torch.long) # indexes for dim=2
v = torch.randn((10, 16, 64)) # to be assigned
t[(i0, slice(None), i2, slice(None))] = v
I am trying to plot the results of PCA of the dataset pima-indians-diabetes.csv. My code shows a problem only in the plotting piece:
import numpy
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import pandas as pd
# Dataset Description:
# 1. Number of times pregnant
# 2. Plasma glucose concentration a 2 hours in an oral glucose tolerance test
# 3. Diastolic blood pressure (mm Hg)
# 4. Triceps skin fold thickness (mm)
# 5. 2-Hour serum insulin (mu U/ml)
# 6. Body mass index (weight in kg/(height in m)^2)
# 7. Diabetes pedigree function
# 8. Age (years)
# 9. Class variable (0 or 1)
path = 'pima-indians-diabetes.data.csv'
dataset = numpy.loadtxt(path, delimiter=",")
X = dataset[:,0:8]
Y = dataset[:,8]
features = ['1','2','3','4','5','6','7','8','9']
df = pd.read_csv(path, names=features)
x = df.loc[:, features].values # Separating out the values
y = df.loc[:,['9']].values # Separating out the target
x = StandardScaler().fit_transform(x) # Standardizing the features
pca = PCA(n_components=2)
principalComponents = pca.fit_transform(x)
# principalDf = pd.DataFrame(data=principalComponents, columns=['pca1', 'pca2'])
# finalDf = pd.concat([principalDf, df[['9']]], axis = 1)
plt.figure()
colors = ['navy', 'turquoise', 'darkorange']
lw = 2
for color, i, target_name in zip(colors, [0, 1, 2], ['Negative', 'Positive']):
plt.scatter(principalComponents[y == i, 0], principalComponents[y == i, 1], color=color, alpha=.8, lw=lw,
label=target_name)
plt.legend(loc='best', shadow=False, scatterpoints=1)
plt.title('PCA of pima-indians-diabetes Dataset')
The error is located at the following line:
Traceback (most recent call last):
File "test.py", line 53, in <module>
plt.scatter(principalComponents[y == i, 0], principalComponents[y == i, 1], color=color, alpha=.8, lw=lw,
IndexError: too many indices for array
Kindly, how to fix this?
As the error indicates some kind of shape/dimension mismatch, a good starting point is to check the shapes of the arrays involved in the operation:
principalComponents.shape
yields
(768, 2)
while
(y==i).shape
(768, 1)
Which leads to a shape mismatch when trying to run
principalComponents[y==i, 0]
as the first array is already multidimensional, therefore the error is indicating that you used too many indices for the array.
You can fix this by forcing the shape of y==i to a 1D array ((768,)), e.g. by changing your call to scatter to
plt.scatter(principalComponents[(y == i).reshape(-1), 0],
principalComponents[(y == i).reshape(-1), 1],
color=color, alpha=.8, lw=lw, label=target_name)
which then creates the plot for me
For more information on the difference between arrays of the shape (R, 1)and (R,) this question on StackOverflow provides a nice starting point.
PyTorch's torch.transpose function only transposes 2D inputs. Documentation is here.
On the other hand, Tensorflow's tf.transpose function allows you to transpose a tensor of N arbitrary dimensions.
Can someone please explain why PyTorch does not/cannot have N-dimension transpose functionality? Is this due to the dynamic nature of the computation graph construction in PyTorch versus Tensorflow's Define-then-Run paradigm?
It's simply called differently in pytorch. torch.Tensor.permute will allow you to swap dimensions in pytorch like tf.transpose does in TensorFlow.
As an example of how you'd convert a 4D image tensor from NHWC to NCHW (not tested, so might contain bugs):
>>> img_nhwc = torch.randn(10, 480, 640, 3)
>>> img_nhwc.size()
torch.Size([10, 480, 640, 3])
>>> img_nchw = img_nhwc.permute(0, 3, 1, 2)
>>> img_nchw.size()
torch.Size([10, 3, 480, 640])
Einops supports verbose transpositions for arbitrary number of dimensions:
from einops import rearrange
x = torch.zeros(10, 3, 100, 100)
y = rearrange(x, 'b c h w -> b h w c')
x2 = rearrange(y, 'b h w c -> b c h w') # inverse to the first
(and the same code works for tensorfow as well)