I just came up with a difference between one line and regular for loop.
As an example;
obs = [6, 12, 8, 10, 20 16]
freq = [5, 4, 3, 2, 1, 5]
data = []
data.extend(obs[i:i+1] * freq[i] for i in range(len(obs)))
outputs
[[6, 6, 6, 6, 6], [12, 12, 12, 12], [8, 8, 8], [10, 10], [20], [16, 16, 16, 16, 16]]
However,
for i in range(len(obs)):
data.extend(obs[i:i+1] * freq[i])
outputs
[6, 6, 6, 6, 6, 12, 12, 12, 12, 8, 8, 8, 10, 10, 20, 16, 16, 16, 16, 16]
Can someone kindly explain what causes this?
extending x by y means appending each entry of y to x.
Since the entries of obs[i:i+1] * freq[i] for i in range(len(obs)) are lists of integers, data.extend(obs[i:i+1] * freq[i] for i in range(len(obs))) will append of lists of integers to data, not integers.
On the other hand, the elements of obs[i:i+1] * freq[i] are integers, and therefore data.extend(obs[i:i+1] * freq[i]) will append integers to data.
This would generate the same output as the one-liner:
obs = [6, 12, 8, 10, 20, 16]
freq = [5, 4, 3, 2, 1, 5]
data = []
for i in range(len(obs)):
data.extend([obs[i:i+1] * freq[i]])
Related
mat = [[0, 1, 2, 3, 4, 5],
[6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]]
Lets say I want to extract upper left 2x2 matrix
[[0, 1,],
[6, 7, ]]
doing mat2=mat[:2][:2] doesnt work.
It extracts the rows correctly but not columns.Seems like I need to loop throughto get the columns.
Additionally I need to do a deepcopy to mat2 suchthat modifying mat2 dont change mat.
This is because [:2] returns a list containing the first 2 elements of your matrix.
For example :-
arr = [[1, 2], [1, 3]]
print(arr[:2]) # will print the first 2 elements of the array, that is [1, 2] and [1, 3], packed into a list. So, Output : [[1, 2], [1, 3]].
In the same way,
mat = [[0, 1, 2, 3, 4, 5],
[6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]]
mat2 = mat[:2] # => [[0, 1, 2, 3, 4, 5], [6, 7, 8, 9, 10, 11]]
# Now, if you again try to get the first 2 elements from mat2 you will get the first 2 elements of mat2, not the first 2 elements of the lists inside mat2.
mat3 = mat2[:2] # => [[0, 1, 2, 3, 4, 5], [6, 7, 8, 9, 10, 11]]
That is where you went wrong, but this concept is quite counter-intuitive, so no worries.
So the solution would be to get the first 2 elements from matrix mat and then loop over its elements and then get the first 2 elements from them.
Therefore, this should work for you:
list(x[:2] for x in mat[:2])
Or, as #warped pointed, if you can use numpy, you can do the following:
import numpy as np
mat = np.array(mat)
mat[:2, :2]
Currently I have a 2D tensor, for each row, I want to insert a new element e before the first index of a specified value v. Additional information: cannot guarantee each row could have a such value. If there isn't, just append the element
Example: Supporse e is 0, v is 10, Given a tensor
[[9, 6, 5, 4, 10],
[8, 7, 3, 5, 5],
[4, 9, 10, 10, 10]]
I want to get
[[9, 6, 5, 4, 0, 10],
[8, 7, 3, 5, 5, 0],
[4, 9, 0, 10, 10, 10]]
Are there some Torch-style ways to do this? The worst case I can treat this as a trivial Python problem but I think the corresponding solution is a little time-consuming.
I haven't yet found a full PyTorch solution. I'll keep looking, but here is somewhere to start:
>>> v, e = 10, 0
>>> v, e = torch.tensor([v]), torch.tensor([e])
>>> x = torch.tensor([[ 9, 6, 5, 4, 10],
[ 8, 7, 3, 5, 5],
[ 4, 9, 10, 10, 10],
[10, 9, 7, 10, 2]])
To deal with the edge case where v is not found in one of the rows you can add a temporary column to x. This will ensure every row has a value v in it. We will use x_ as a helper tensor:
>>> x_ = torch.cat([x, v.repeat(x.size(0))[:, None]], axis=1)
tensor([[ 9, 6, 5, 4, 10, 10],
[ 8, 7, 3, 5, 5, 10],
[ 4, 9, 10, 10, 10, 10],
[10, 9, 7, 10, 2, 10]])
Find the indices of the first value v on each row:
>>> bp = (x_ == v).int().argmax(axis=1)
tensor([4, 5, 2, 0])
Finally, the easiest way to insert values at different positions in each row is with a list comprehension:
>>> torch.stack([torch.cat([xi[:bpi], e, xi[bpi:]]) for xi, bpi in zip(x, bp)])
tensor([[ 9, 6, 5, 4, 0, 10],
[ 8, 7, 3, 5, 5, 0],
[ 4, 9, 0, 10, 10, 10],
[ 0, 10, 9, 7, 10, 2]])
Edit - If v cannot occur in the first position, then no need for x_:
>>> x
tensor([[ 9, 6, 5, 4, 10],
[ 8, 7, 3, 5, 5],
[ 4, 9, 10, 10, 10]])
>>> bp = (x == v).int().argmax(axis=1) - 1
>>> torch.stack([torch.cat([xi[:bpi], e, xi[bpi:]]) for xi, bpi in zip(x, bp)])
tensor([[ 9, 6, 5, 0, 4, 10],
[ 8, 7, 3, 5, 0, 5],
[ 4, 0, 9, 10, 10, 10]])
Here is my matrices and codeline:
d = np.array([[1,2,3],[6,7,8],[11,12,13],
[16,17,18]])
e = np.array([[ 4, 5],[ 9, 10],[14, 15],[19, 20]])
np.concatenate(d,e)
and this is the error that I get:
TypeError: only integer scalar arrays can be converted to a scalar index
You have a syntax mistake in np.concatenate(d,e), the syntax requires d and e to be in a tuple, like: np.concatenate((d,e)). I tested it, and axis=1 is also required for it to work.
np.concatenate((d, e), axis=1)
is the solution
Since those arrays have different dimensions you should specify the axis concatenate you what like the follow:
1) np.concatenate((d,e), axis=1)
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])
or
2)np.concatenate((d,e), axis=None)
array([ 1, 2, 3, 6, 7, 8, 11, 12, 13, 16, 17, 18, 4, 5, 9, 10, 14,
15, 19, 20])
I have a 2d numpy array as such:
import numpy as np
a = np.arange(20).reshape((2,10))
# array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
# [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]])
I want to swap pairs of elements in each row. The desired output looks like this:
# array([[ 9, 0, 2, 1, 4, 3, 6, 5, 8, 7],
# [19, 10, 12, 11, 14, 13, 16, 15, 18, 17]])
I managed to find a solution in 1d:
a = np.arange(10)
# does the job for all pairs except the first
output = np.roll(np.flip(np.roll(a,-1).reshape((-1,2)),1).flatten(),2)
# first pair done manually
output[0] = a[-1]
output[1] = a[0]
Any ideas on a "numpy only" solution for the 2d case ?
Owing to the first pair not exactly subscribing to the usual pair swap, we can do that separately. For the rest, it would relatively straight-forward with reshaping to split axes and flip axis. Hence, it would be -
In [42]: a # 2D input array
Out[42]:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]])
In [43]: b2 = a[:,1:-1].reshape(a.shape[0],-1,2)[...,::-1].reshape(a.shape[0],-1)
In [44]: np.hstack((a[:,[-1,0]],b2))
Out[44]:
array([[ 9, 0, 2, 1, 4, 3, 6, 5, 8, 7],
[19, 10, 12, 11, 14, 13, 16, 15, 18, 17]])
Alternatively, stack and then reshape+flip-axis -
In [50]: a1 = np.hstack((a[:,[0,-1]],a[:,1:-1]))
In [51]: a1.reshape(a.shape[0],-1,2)[...,::-1].reshape(a.shape[0],-1)
Out[51]:
array([[ 9, 0, 2, 1, 4, 3, 6, 5, 8, 7],
[19, 10, 12, 11, 14, 13, 16, 15, 18, 17]])
When does map modify an array in place? I know the preferred way to iterate over an array is with a list comprehension, but I'm preparing an algorithm for ipyparallel, which apparently uses the map function. Each row of my array is a set of model inputs, and I want to use map, ultimately in parallel, to run the model for each row. I'm using Python 3.4.5 and Numpy 1.11.1. I need these versions for compatibility with other packages.
This simple example creates a list and leaves the input array intact, as I expected.
grid = np.arange(25).reshape(5,5)
grid
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
def f(g):
return g + 1
n = list(map(f, grid))
grid
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
But when the function modifies a slice of the input row, the array is modified in place. Can anyone explain this behavior?
def f(g):
g[:2] = g[:2] + 1
return g
n = list(map(f, grid))
grid
array([[ 1, 2, 2, 3, 4],
[ 6, 7, 7, 8, 9],
[11, 12, 12, 13, 14],
[16, 17, 17, 18, 19],
[21, 22, 22, 23, 24]])