I'm trying to tile a length 18 1 hot vector into a 40x40 grid.
Looking at pytorch docs, expand dims seems to be what i need.
But I cannot get it to work. Any idea what I'm doing wrong?
one_hot = torch.zeros(18).unsqueeze(0)
one_hot[0,1] = 1.0
one_hot
tensor([[0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
one_hot.expand(-1,-1,40,40)
Traceback (most recent call last):
File "<input>", line 1, in <module>
RuntimeError: The expanded size of the tensor (40) must match the existing size (18) at non-singleton dimension 3
I'm expecting a tensor of shape (1, 18, 40,40)
expand works along singleton dimensions of the input tensor. In your example, you are trying to expand a 1-by-18 tensor along its (non-existent) third and fourth dimensions - this is why you are getting an error. The only singleton dimension (=dimension with size==1) you have is the first dimension.
fix
one_hot = torch.zeros(1,18,1,1, dtype=torch.float) # create the tensor with all singleton dimensions in place
one_hot[0,1,0,0] = 1.
one_hot.expand(-1,-1,40,40)
Related
I'm new on PyTorch and I'm trying to code with it
so I have a function called OH which tack a number and return a vector like this
def OH(x,end=10,l=12):
x = T.LongTensor([[x]])
end = T.LongTensor([[end]])
one_hot_x = T.FloatTensor(1,l)
one_hot_end = T.FloatTensor(1,l)
first=one_hot_x.zero_().scatter_(1,x,1)
second=one_hot_end.zero_().scatter_(1,end,1)
vector=T.cat((one_hot_x,one_hot_end),dim=1)
return vector
OH(0)
output:
tensor([[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 1., 0.]])
now I have a NN that takes this output and return number but this warning always appear in my compiling
online.act(OH(obs))
output:
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:17: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
4
I tried to to use online.act(OH(obs).clone().detach()) but it give me the same warning
and the code works fine and give good results but I need to understand this warning
Edit
the following is my NN that has the act function
class Network(nn.Module):
def __init__(self,lr,n_action,input_dim):
super(Network,self).__init__()
self.f1=nn.Linear(input_dim,128)
self.f2=nn.Linear(128,64)
self.f3=nn.Linear(64,32)
self.f4=nn.Linear(32,n_action)
#self.optimizer=optim.Adam(self.parameters(),lr=lr)
#self.loss=nn.MSELoss()
self.device=T.device('cuda' if T.cuda.is_available() else 'cpu')
self.to(self.device)
def forward(self,x):
x=F.relu(self.f1(x))
x=F.relu(self.f2(x))
x=F.relu(self.f3(x))
x=self.f4(x)
return x
def act(self,obs):
state=T.tensor(obs).to(device)
actions=self.forward(state)
action=T.argmax(actions).item()
return action
the problem is that you are receiving a tensor on the act function on the Network and then save it as a tensor
just remove the tensor in the action like this
def act(self,obs):
#state=T.tensor(obs).to(device)
state=obs.to(device)
actions=self.forward(state)
action=T.argmax(actions).item()
I'm trying to learn RNN and Pytorch.
So I saw some codes for RNN where in the forward probagation method, they did a check like this:
def forward(self, inputs, hidden):
if inputs.is_cuda:
device = inputs.get_device()
else:
device = torch.device("cpu")
embed_out = self.embeddings(inputs)
logits = torch.zeros(self.seq_len, self.batch_size, self.vocab_size).to(device)
I think the point of the check is to see if we can run the code on faster GPU instead of CPU?
To understand the code a bit more, I did the following:
ex= torch.zeros(3,10,5)
ex1= torch.tensor(np.array([[0,0,0,1,0], [1,0,0,0,0],[0,1,0,0,0]]))
print(ex)
print("device is")
print(ex1.get_device())
print(ex.to(ex1.get_device()))
And the output was:
...
[[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]]])
device is
-1
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-2-b09342e2ba0f> in <module>()
67 print("device is")
68 print(ex1.get_device())
---> 69 print(ex.to(ex1.get_device()))
RuntimeError: Device index must not be negative
I don't understand the "device" in the code and I don't understand the .to(device) method. Can you help me understand it?
This code is deprecated. Just do:
def forward(self, inputs, hidden):
embed_out = self.embeddings(inputs)
logits = torch.zeros((self.seq_len, self.batch_size, self.vocab_size), device=inputs.device)
Note that to(device) is cost-free if the tensor is already on the requested device. And do not use get_device() but rather device attribute. It is working fine with cpu and gpu out of the box.
Also, note that torch.tensor(np.array(...)) is a bad practice for several reasons. First, to convert numpy array to torch tensor either use as_tensor or from_numpy. THen, you will get a tensor with default numpy dtype instead of torch. In this case it is the same (int64), but for float it would be different. Finally, torch.tensor can be initialized using a list, just as numpy array, so you can get rid of numpy completely and call torch directly.
Despite the overwhelming amount of posts on fitting Poisson distribution onto a histogram, having followed all of them, none of them seems to work for me.
I'm looking to fit a poisson distribution on this histogram which I've plotted as such:
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
from scipy.misc import factorial
def poisson(t, rate, scale): #scale is added here so the y-axis
# of the fit fits the height of histogram
return (scale*(rate**t/factorial(t))*np.exp(-rate))
lifetimes = 1/np.random.poisson((1/550e-6), size=100000)
hist, bins = np.histogram(lifetimes, bins=50)
width = 0.8*(bins[1]-bins[0])
center = (bins[:-1]+bins[1:])/2
plt.bar(center, hist, align='center', width=width, label = 'Normalised data')
popt, pcov = curve_fit(poisson, center, hist, bounds=(0.001, [2000, 7000]))
plt.plot(center, poisson(center, *popt), 'r--', label='Poisson fit')
# import pdb; pdb.set_trace()
plt.legend(loc = 'best')
plt.tight_layout()
The histogram I get looks like this:
I gave the guess of scale as 7000 to scale the distribution to the same height as the y-axis of the histogram I plotted and a guess of 2000 as the rate parameter since it's 2000 > 1/550e-6. As you can see the fitted red dotted line is 0 at every point. Weirdly pdb.set_trace() tells me that the poisson(center, *popt) gives me a list of 0 values.
126 plt.plot(center, poisson(center, *popt), 'r--', label='Poisson fit')
127 import pdb; pdb.set_trace()
--> 128 plt.legend(loc = 'best')
129 plt.tight_layout()
130
ipdb>
ipdb> poisson(center, *popt)
array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
Which doesn't make sense. What I want is to fit a poisson distribution on the histogram such that it finds the best coefficient of the poisson distribution equation. I suspected that it might be have to do with because I am plotting histogram of lifetimes instead, which is technically randomly sampled data from the inverse of the poisson distribution. So I tried to compute the jacobian of the distribution so I can make a change of variables but it still won't work. I feel like I'm missing something here that's not coding but rather mathematics related.
You're calculation is rounding to zero. With a rate of 2000 and scale of 7000 your poisson formula is reduced to:
7000 * 2000^t/(e^(2000) * t!)
Using Stirling's approximation t! ~ (2*pi*t)^(1/2) * (t/e)^t you get:
[7000 * 2000^t] / [Sqrt(2*pi*t) * e^(2000-t) * (t^t)] ~ poisson(t)
I used python to get the first couple values of poisson(t):
poisson(1) -> 0
poisson(2) -> 0
poisson(3) -> 0
Using wolfram alpha you find that the derivative of the denominator greater than the derivative of the numerator for all real numbers greater than zero. Therefore, poisson(t) is approaching zero as t gets larger.
This means that no matter what t is, if your rate is 2000, the poisson function will return 0.
Sorry for the formatting. They wont let me post TeX yet.
I can apply the following code to an array.
from numpy import *
A = eye(4)
A[A[:,1] > 0.5,:]
But How can I apply the similar method to a mat?
A = mat(eye(4))
A[A[:,1] > 0.5,:]
I know the above code is wrong, but what should I do?
The problem is that, when A is a numpy.matrix, A[:,1] returns a 2-d matrix, and therefore A[:,1] > 0.5 is also 2-d. Anything that makes this expression look like the same thing that is created when A is an ndarray will work. For example, you can write A.A[:,1] > 0.5 (the .A attribute returns an ndarray view of the matrix), or (A[:,1] > 0.5).A1 (the A1 attribute returns a flatten ndarray).
For example,
In [119]: A
Out[119]:
matrix([[ 1., 0., 0., 0.],
[ 0., 1., 0., 0.],
[ 0., 0., 1., 0.],
[ 0., 0., 0., 1.]])
In [120]: A[(A[:, 1] > 0.5).A1,:]
Out[120]: matrix([[ 0., 1., 0., 0.]])
In [121]: A[A.A[:, 1] > 0.5,:]
Out[121]: matrix([[ 0., 1., 0., 0.]])
Because of quirks like these, I (and many others) recommend avoiding the numpy.matrix class. Most code can be written just as easily by using ndarrays throughout.
Data is shared variables. I want to get the predictation result in csv format. Below is the code.
It throws an error. How to fix? Thank you for your help!
TypeError: ('Bad input argument to theano function with name "4.py:305" at index
0(0-based)', 'Expected an array-like object,
but found a Variable: maybe you are trying to call a function on a (possibly shared)
variable instead of a numeric array?')
test_model = theano.function(
inputs=[index],
outputs=classifier.errors(y),
givens={
x: test_set_x[index * batch_size:(index + 1) * batch_size],
y: test_set_y[index * batch_size:(index + 1) * batch_size]
}
)
def make_submission_csv(predict, is_list=False):
if is_list:
df = pd.DataFrame({'Id': range(1, 101), 'Label': predict})
df.to_csv("submit.csv", index=False)
return
pred = []
for i in range(100):
pred.append(test_model(test.values[i]))
df = pd.DataFrame({'Id': range(1, 101), 'Label': pred})
df.to_csv("submit.csv", index=False)
make_submission_csv(np.argmax(test_model(test_set_x), axis=1), is_list=True)
And more information about "index".
index = T.iscalar()
x = T.matrix('x')
y = T.ivector('y')
when enter:
test_set_x.get_value(borrow=True)
The console shows:
array([[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
...,
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.]], dtype=float32)
When enter:
test_model(test_set_x.get_value())
It throws an error:
TypeError: ('Bad input argument to theano function with name "4.py:311" at index 0(0-based)', 'TensorType(int32, scalar) cannot store a value of dtype float32 without risking loss of precision.
Your test_model function has a single input value,
inputs=[index],
Your pasted code doesn't show the creation of the variable index but my guess is that it's a Theano symbolic scalar with an integer type. If so, you need to call the compiled function with a single integer input, for example
test_model(1)
You are trying to call test_model(test_set_x) which doesn't work because test_set_x is (again probably) a shared variable, not the integer index the function is expecting.
Note that the tutorial code does this:
test_losses = [test_model(i) for i in xrange(n_test_batches)]