Related
I used torchtext vocab to convert the text to index, but which function should I use to make all the index list be the same length before I send them to the net?
For example I have 2 texts:
I am a good man
I would like a coffee please
After vocab:
[1, 3, 2, 5, 7]
[1, 9, 6, 2, 4, 8]
And what I want is:
[1, 3, 2, 5, 7, 0]
[1, 9, 6, 2, 4, 8]
It is easy to understand by looking at the following example.
Code:
import torch
v = [
[0,2],
[0,1,2],
[3,3,3,3]
]
torch.nn.utils.rnn.pad_sequence([torch.tensor(p) for p in v], batch_first=True)
Result:
tensor([[0, 2, 0, 0],
[0, 1, 2, 0],
[3, 3, 3, 3]])
It seems PyTorch unfold and as_stride are doing the same thing but for the former, you cannot control the tensor output size.
import torch
import torch.nn as nn
x = torch.arange(0, 10)
x1 = x.unfold(0, 3, 1)
x2 = x.as_strided((8,3), (1,1))
print(f'x1 = {x1}')
print(f'x2 = {x2}')
output:
x1 = tensor([[0, 1, 2],
[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7],
[6, 7, 8],
[7, 8, 9]])
x2 = tensor([[0, 1, 2],
[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7],
[6, 7, 8],
[7, 8, 9]])
Then is there any situation that you should use unfold instead of as_stride and vice versa?
Can anyone help me. This is what i want to do.
x = [[1,2,3,4,5],[6,7,8,9,10]]
y= [0,1]
desired output = [
[[1,2,3,4,5],[0,1]],
[[6,7,8,9,10],[0,1]]
]
I try putting it in a for loop
>>> x = [[1,2,3,4,5],[6,7,8,9,10]]
>>> for value in x:
... a = []
... a += ([x,y])
... print(a)
...
[[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]], [0, 1]]
[[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]], [0, 1]]
I also tried doing this
>>> for value in x:
... a = []
... a += ([x,y])
... print(a)
...
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
Thank you for helping. I need it for putting label on my data for neural networks.
You can use a list comprehension, and iterate over each sublist in x. Since you're inserting y into different sublists, you might want to insert a copy of the list, not the original.
[[i, y[:]] for i in x]
Or,
[[i, y.copy()] for i in x]
[[[1, 2, 3, 4, 5], [0, 1]], [[6, 7, 8, 9, 10], [0, 1]]]
The copy is done as a safety precaution. To understand why, consider an example,
z = [[i, y] for i in x] # inserting y (reference copy)
y[0] = 12345
print(z)
[[[1, 2, 3, 4, 5], [12345, 1]], [[6, 7, 8, 9, 10], [12345, 1]]] # oops
Modifying the original y or the y in any other sublist will reflect changes across all sublists. You can prevent that by inserting a copy instead, which is what I've done at the top.
Try this:
for i in range(len(x)):
z[i] = [x[i],y];
How do I slice or index this list in order to get the answer below? I've tried doing multiple methods of slicing and nothing has worked for me.
L = [0, [], [1,2,3,4], [[5],[6,7]], [8,9,10]]
newL = [L[0],L[2][1],L[2][2],L[3][0]]
Answer: [0, 2, 3, [5 ,6], 8, 10]
newL is what I have so far, but I can't seem to get the [6,7] split in the nested list.
We start with:
L = [0, [], [1, 2, 3, 4], [[5], [6, 7]], [8, 9, 10]]
We want:
[0, 2, 3, [5, 6], 8, 10]
Let's start from the farthest inside. We need [5, 6]. These are both buried:
>>> L[3][0][0]
5
>>> L[3][1][0]
6
>>> [L[3][0][0], L[3][1][0]]
[5, 6]
One layer farther out we need 2, 3:
>>> L[2][1]
2
>>> L[2][2]
3
Now put it together:
>>> [L[0], L[2][1], L[2][2], [L[3][0][0], L[3][1][0]], L[4][0], L[4][2]]
[0, 2, 3, [5, 6], 8, 10]
Suppose I have the following numpy.array:
In[]: x
Out[]:
array([[1, 2, 3, 4, 5],
[5, 2, 4, 1, 5],
[6, 7, 2, 5, 1]], dtype=int16)
In[]: y
Out[]:
array([[-3, -4],
[-4, -1]], dtype=int16)
I want to replace a sub array of x by y and tried the following:
In[]: x[[0,2]][:,[1,3]]= y
Ideally, I wanted this to happen:
In[]: x
Out[]:
array([[1, -3, 3, -4, 5],
[5, 2, 4, 1, 5],
[6, -4, 2, -1, 1]], dtype=int16)
The assignment line doesn't give me any error, but when I check the output of x
In[]: x
I find that x hasn't changed, i.e. the assignment didn't happen.
How can I make that assignment? Why did the assignment didn't happen?
The the "fancy indexing" x[[0,2]][:,[1,3]] returns a copy of the data. Indexing with slices returns a view. The assignment does happen, but to a copy (actually a copy of a copy of...) of x.
Here we see that the indexing returns a copy:
>>> x[[0,2]]
array([[1, 2, 3, 4, 5],
[6, 7, 2, 5, 1]], dtype=int16)
>>> x[[0,2]].base is x
False
>>> x[[0,2]][:, [1, 3]].base is x
False
>>>
Now you can use fancy indexing to set array values, but not when you nest the indexing.
You can use np.ix_ to generate the indices and perform the assignment:
>>> x[np.ix_([0, 2], [1, 3])]
array([[2, 4],
[7, 5]], dtype=int16)
>>> np.ix_([0, 2], [1, 3])
(array([[0],
[2]]), array([[1, 3]]))
>>> x[np.ix_([0, 2], [1, 3])] = y
>>> x
array([[ 1, -3, 3, -4, 5],
[ 5, 2, 4, 1, 5],
[ 6, -4, 2, -1, 1]], dtype=int16)
>>>
You can also make it work with broadcasted fancy indexing (if that's even the term) but it's not pretty
>>> x[[0, 2], np.array([1, 3])[..., None]] = y
>>> x
array([[ 1, -3, 3, -4, 5],
[ 5, 2, 4, 1, 5],
[ 6, -4, 2, -1, 1]], dtype=int16)
By the way, there is some interesting discussion at the moment on the NumPy Discussion mailing list on better support for "orthogonal" indexing so this may become easier in the future.