Here is my matrices and codeline:
d = np.array([[1,2,3],[6,7,8],[11,12,13],
[16,17,18]])
e = np.array([[ 4, 5],[ 9, 10],[14, 15],[19, 20]])
np.concatenate(d,e)
and this is the error that I get:
TypeError: only integer scalar arrays can be converted to a scalar index
You have a syntax mistake in np.concatenate(d,e), the syntax requires d and e to be in a tuple, like: np.concatenate((d,e)). I tested it, and axis=1 is also required for it to work.
np.concatenate((d, e), axis=1)
is the solution
Since those arrays have different dimensions you should specify the axis concatenate you what like the follow:
1) np.concatenate((d,e), axis=1)
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])
or
2)np.concatenate((d,e), axis=None)
array([ 1, 2, 3, 6, 7, 8, 11, 12, 13, 16, 17, 18, 4, 5, 9, 10, 14,
15, 19, 20])
Related
mat = [[0, 1, 2, 3, 4, 5],
[6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]]
Lets say I want to extract upper left 2x2 matrix
[[0, 1,],
[6, 7, ]]
doing mat2=mat[:2][:2] doesnt work.
It extracts the rows correctly but not columns.Seems like I need to loop throughto get the columns.
Additionally I need to do a deepcopy to mat2 suchthat modifying mat2 dont change mat.
This is because [:2] returns a list containing the first 2 elements of your matrix.
For example :-
arr = [[1, 2], [1, 3]]
print(arr[:2]) # will print the first 2 elements of the array, that is [1, 2] and [1, 3], packed into a list. So, Output : [[1, 2], [1, 3]].
In the same way,
mat = [[0, 1, 2, 3, 4, 5],
[6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]]
mat2 = mat[:2] # => [[0, 1, 2, 3, 4, 5], [6, 7, 8, 9, 10, 11]]
# Now, if you again try to get the first 2 elements from mat2 you will get the first 2 elements of mat2, not the first 2 elements of the lists inside mat2.
mat3 = mat2[:2] # => [[0, 1, 2, 3, 4, 5], [6, 7, 8, 9, 10, 11]]
That is where you went wrong, but this concept is quite counter-intuitive, so no worries.
So the solution would be to get the first 2 elements from matrix mat and then loop over its elements and then get the first 2 elements from them.
Therefore, this should work for you:
list(x[:2] for x in mat[:2])
Or, as #warped pointed, if you can use numpy, you can do the following:
import numpy as np
mat = np.array(mat)
mat[:2, :2]
There exist an numpy ndarry A of shape [100,50, 5], and I want to expand A as follows. A will be appended with an one-dimensional array of shape (50, ). The resulting A will have shape [100,50,6].
The element of this one-dimensional array is based on the array in the original ndarray, i.e., A[:,:,4] in terms of a given formula, i.e., A[:,i,5]=A[:,i,4]*B[i]+5 for i = 0:49 Here A[:,:,5] corresponds to the added one-dimensional array. B is another array working as weight.
Besides using a for loop to write this function, how to fullfill this task in a vectorized/efficient way leveraging numpy operation
Make 2 arrays - with sizes that we can look at:
In [371]: A = np.arange(24).reshape(2,3,4); B = np.array([10,20,30])
Due to broadcasting we can add a (3,) array to (2,3) array
In [372]: A[:,:,-1]+B
Out[372]:
array([[13, 27, 41],
[25, 39, 53]])
we can then convert that to (2,3,1) array:
In [373]: (A[:,:,-1]+B)[:,:,None]
Out[373]:
array([[[13],
[27],
[41]],
[[25],
[39],
[53]]])
In [374]: A
Out[374]:
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
and join them on the last axis:
In [375]: np.concatenate((A, Out[373]), axis=-1)
Out[375]:
array([[[ 0, 1, 2, 3, 13],
[ 4, 5, 6, 7, 27],
[ 8, 9, 10, 11, 41]],
[[12, 13, 14, 15, 25],
[16, 17, 18, 19, 39],
[20, 21, 22, 23, 53]]])
Or we can make a target array of the right size, and copy values to it:
In [376]: A1 = np.zeros((2,3,5),int)
In [377]: A1[:,:,:-1]=A
In [379]: A1[:,:,-1]=Out[372]
I want to access all rows except for the first 3 for the specific columns at index 1, 2, 4, 5, 7, 8, 10, 11, 13, 14 of a csv file. How can I do this? All examples I have found show how to slice (for example 1:14 but I do not want all columns in between but specific ones.
When I try:
p = df[3:, [1, 2, 4, 5, 7, 8, 10, 11, 13, 14]]
I get an error:
p = df[3:, [1, 2, 4, 5, 7, 8, 10, 11, 13, 14]]
File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 2139, in __getitem__
return self._getitem_column(key)
File "/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py", line 2146, in _getitem_column
return self._get_item_cache(key)
File "/usr/local/lib/python3.5/dist-packages/pandas/core/generic.py", line 1840, in _get_item_cache
res = cache.get(item)
TypeError: unhashable type: 'slice'
and it does not work with the notation p = df[[3:], [1, 2, 4, 5, 7, 8, 10, 11, 13, 14]]
IIUC you need DataFrame.iloc for filter by positions here all rows without first 3 and columns names by positions:
df.iloc[3:, [1, 2, 4, 5, 7, 8, 10, 11, 13, 14]]
p = df.iloc[[3:],[1, 2, 4, 5, 7, 8, 10, 11, 13, 14]]
you were mostly right except you just had to close the square brackets after "3:".
and using loc, iloc is recommended for indexing.
I have a 2d numpy array as such:
import numpy as np
a = np.arange(20).reshape((2,10))
# array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
# [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]])
I want to swap pairs of elements in each row. The desired output looks like this:
# array([[ 9, 0, 2, 1, 4, 3, 6, 5, 8, 7],
# [19, 10, 12, 11, 14, 13, 16, 15, 18, 17]])
I managed to find a solution in 1d:
a = np.arange(10)
# does the job for all pairs except the first
output = np.roll(np.flip(np.roll(a,-1).reshape((-1,2)),1).flatten(),2)
# first pair done manually
output[0] = a[-1]
output[1] = a[0]
Any ideas on a "numpy only" solution for the 2d case ?
Owing to the first pair not exactly subscribing to the usual pair swap, we can do that separately. For the rest, it would relatively straight-forward with reshaping to split axes and flip axis. Hence, it would be -
In [42]: a # 2D input array
Out[42]:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]])
In [43]: b2 = a[:,1:-1].reshape(a.shape[0],-1,2)[...,::-1].reshape(a.shape[0],-1)
In [44]: np.hstack((a[:,[-1,0]],b2))
Out[44]:
array([[ 9, 0, 2, 1, 4, 3, 6, 5, 8, 7],
[19, 10, 12, 11, 14, 13, 16, 15, 18, 17]])
Alternatively, stack and then reshape+flip-axis -
In [50]: a1 = np.hstack((a[:,[0,-1]],a[:,1:-1]))
In [51]: a1.reshape(a.shape[0],-1,2)[...,::-1].reshape(a.shape[0],-1)
Out[51]:
array([[ 9, 0, 2, 1, 4, 3, 6, 5, 8, 7],
[19, 10, 12, 11, 14, 13, 16, 15, 18, 17]])
When does map modify an array in place? I know the preferred way to iterate over an array is with a list comprehension, but I'm preparing an algorithm for ipyparallel, which apparently uses the map function. Each row of my array is a set of model inputs, and I want to use map, ultimately in parallel, to run the model for each row. I'm using Python 3.4.5 and Numpy 1.11.1. I need these versions for compatibility with other packages.
This simple example creates a list and leaves the input array intact, as I expected.
grid = np.arange(25).reshape(5,5)
grid
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
def f(g):
return g + 1
n = list(map(f, grid))
grid
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
But when the function modifies a slice of the input row, the array is modified in place. Can anyone explain this behavior?
def f(g):
g[:2] = g[:2] + 1
return g
n = list(map(f, grid))
grid
array([[ 1, 2, 2, 3, 4],
[ 6, 7, 7, 8, 9],
[11, 12, 12, 13, 14],
[16, 17, 17, 18, 19],
[21, 22, 22, 23, 24]])