I have to numpy.ndarray A & B which are of the following shape
A=(500000,784),B =(500000,).I need to add these 2 arrays in a way that the array B , which has labels gets added as the 785th column in the array without changing any sequence in its row- wise data.
i.e, A becomes of shape (500000,785).
np.append(A.T,[B.T], axis=0).T
For example:
A = np.array([[1,2,3],[4,5,6],[7,8,9],[10,9,11]])
B = np.array([4,5,3,6])
np.append(A.T,[B.T], axis=0).T
Output:
array([[ 1, 2, 3, 4],
[ 4, 5, 6, 5],
[ 7, 8, 9, 3],
[10, 9, 11, 6]])
Related
I would like to create a 4x4 tensor that is composed of four smaller 2x2 tensors in this manner:
The tensor I would like to create:
in_t = torch.tensor([[14, 7, 6, 2],
[ 4, 8, 11, 1],
[ 3, 5, 9, 10],
[12, 15, 16, 13]])
I would like to create this tensor composed from these four smaller tensors:
a = torch.tensor([[14, 7], [ 4, 8]])
b = torch.tensor([[6, 2], [11, 1]])
c = torch.tensor([[3, 5], [12, 15]])
d = torch.tensor([[9, 10], [16, 13]])
I have tried to use torch.cat like this:
mm_ab = torch.cat((a,b,c,d), dim=0)
but I end up with an 8x2 tensor.
You can control the layout of your tensor and achieve the desired result with a combination of torch.transpose and torch.reshape. You can perform an outer transpose followed by an inner transpose:
>>> stack = torch.stack((a,b,c,d))
tensor([[[14, 7],
[ 4, 8]],
[[ 6, 2],
[11, 1]],
[[ 3, 5],
[12, 15]],
[[ 9, 10],
[16, 13]]])
Reshape-tranpose-reshape-transpose-reshape:
>>> stack.reshape(4,2,-1).transpose(0,1).reshape(-1,2,4).transpose(0,1).reshape(-1,4)
tensor([[14, 7, 6, 2],
[ 4, 8, 11, 1],
[ 3, 5, 9, 10],
[12, 15, 16, 13]])
Essentially, reshapes allow you to group and view your tensor differently while transpose operation will alter its layout (it won't remain contiguous) meaning you can achieve the desired output.
If you concatenate all your tensors this way below, you will get exactly your output:
tensor a
tensor b
tensor c
tensor d
You really started with a good and easy approach, this is the completion of your attempt:
p1 = torch.concat((a,b),axis=1)
p2 = torch.concat((c,d),axis=1)
p3 = torch.concat((p1,p2),axis=0)
print(p3)
#output
tensor([[14, 7, 6, 2],
[ 4, 8, 11, 1],
[ 3, 5, 9, 10],
[12, 15, 16, 13]])
I'm trying to find the first occurrence of any row in an array in which either column has a number that has changed since the last time it appeared. Given the array below:
import numpy as np
arr = np.array([[1, 11], [2, 21], [3, 31], [4, 41], [1, 11], [2, 21], [3, 31], [4, 42]])
The output I'm looking for would look like:
subArr = [[1, 11]
[2, 21]
[3, 31]
[4, 41]
[4, 42]]
In the actual problem, the numbers are not as sequential as they appear here and cannot be predicted in advance. I've tried finding the first instance in an array, using multiple conditions, trying to get the first element in a 2-D array, and accessing the ith column. Although some of these were helpful but I can't get it do all the things I want. I tried:
subArr = arr[np.unique(np.logical_and(arr[:,0][0], arr[:,1][0]))]
which didn't work. I also tried:
subArr = arr[(arr[:,0][0]) & (arr[:,1][0])]
I'm sure it's just a matter of getting the syntax right but I can't figure out what I'm missing. Any help would be greatly appreciated.
Using:
Python 3.6
Numpy 1.18.1
Use the axis parameter of numpy.unique:
In [16]: arr
Out[16]:
array([[ 1, 11],
[ 2, 21],
[ 3, 31],
[ 4, 41],
[ 1, 11],
[ 2, 21],
[ 3, 31],
[ 4, 42]])
In [17]: np.unique(arr, axis=0)
Out[17]:
array([[ 1, 11],
[ 2, 21],
[ 3, 31],
[ 4, 41],
[ 4, 42]])
The returned values are copies of the rows from the original array, so it doesn't really make sense to ask if a row in the output corresponds to the first occurrence of the same values in the input.
If you need to know the indices of the first occurrence of each unique row in the input, you can add the argument return_index. When you do this, unique ensures that the index will be that of the first occurrence of the corresponding unique value:
In [51]: values, indices = np.unique(arr, return_index=True, axis=0)
In [52]: values
Out[52]:
array([[ 1, 11],
[ 2, 21],
[ 3, 31],
[ 4, 41],
[ 4, 42]])
In [53]: indices
Out[53]: array([0, 1, 2, 3, 7]
I need to do some calculations with a NetCDF file.
So I have two variables with following dimensions and sizes:
A [time | 1] x [lev | 12] x [lat | 84] x [lon | 228]
B [lev | 12]
What I need is to produce a new array, C, that is shaped as (1,12,84,228) where B contents are propagated to all dimensions of A.
Usually, this is easily done in NCL with the conform function. I am not sure what is the equivalent of this in Python.
Thank you.
The numpy.broadcast_to function can do something like this, although in this case it does require B to have a couple of extra trailing size 1 dimension added to it to satisfy the numpy broadcasting rules
>>> import numpy
>>> B = numpy.arange(12).reshape(12, 1, 1)
>>> B
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
>>> B = B.reshape(12, 1, 1)
>>> B.shape
(12, 1, 1)
>>> C = numpy.broadcast_to(b, (1, 12, 84, 228))
>>> C.shape
(1, 12, 84, 228)
>>> C[0, :, 0, 0]
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
>>> C[-1, :, -1, -1]
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
I was wondering if there was a reasonably efficient way of sampling from a 2d numpy array. If I have a generic array:
dims = (4,4)
test_array = np.arange(np.prod(dims)).reshape(*dims)
test_array
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
Then I'd like to randomly set, say, two elements from it to a specific value (let's say 100). I've tried creating an indexing array and then applying that:
sample_from = np.random.randint(low=0, high=5, size=(2,2))
sample_from
array([[0, 2],
[1, 1]])
But if I try using this to index, it gives me a slightly unexpected answer:
test_array[sample_from]
array([[[ 0, 1, 2, 3],
[ 8, 9, 10, 11]],
[[ 4, 5, 6, 7],
[ 4, 5, 6, 7]]])
What I would have expected (and the kind of result I'd like) is if I'd just entered the indexing array directly:
test_array[[0,2],[1,1]] = 100
test_array
giving:
array([[ 0, 100, 2, 3],
[ 4, 5, 6, 7],
[ 8, 100, 10, 11],
[ 12, 13, 14, 15]])
Any help gratefully received.
You could use np.random.choice + np.unravel_index to assign directly to your array.
test_array[
np.unravel_index(np.random.choice(np.prod(dims), 2, replace=False), dims)
] = 100
I have an assignment to do, which is:
Write a function transpose which takes in a matrix and transposes it. Basically, this converts a m x n matrix into a n x m matrix.
I wrote a code which seems sensible, but it doesnt get me the result I want. Can anyone point out what is wrong with my code?
def transpose(matrix):
new_matrix=[[]]*len(matrix[0])
for row in matrix:
i=0
for j in row:
new_matrix[i]+=[j]
i+=1
return new_matrix
Test case:
print(transpose([[ 1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]))
If you use the * to multiply some values in a list-initialisation, be careful. You might end up with references that point multiple times to the same value:
l = [ [] ]*3
print(l)
l[1].append(34) # change only "the first" list by appending smth
print(l)
Output:
[[], [], []]
[[34], [34], [34]] # they are all "the same data" reference
There is an built-in zip() that does exactly your transposing:
l = [[ 1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
l_t = [ list(x) for x in zip(*l)] # one-line solutions for transposing ;)
print(l)
print(l_t) # transposed
Zip has the limitation that it only works to the length of the smallest sublists - yours are all equal so all is fine.
Output:
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
If you ever need a zip that takes the longest list, itertools.zip_longest(..) can be used, it takes a default param that is substituted for any shorter list-items that are not there.
Btw. just list(zip(l)) looks like this: [(1,5,9),(2,6,10),(3,7,11),(4,8,12)] - it create tuples over the same indexes of the parts of the iterable you put into it.
By hand:
l = [[ 1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
l2 = []
for colIdx in range(len(l[0])): # 0-3 iterate over the inner indexes first
newRow = []
for rowIdx in range(len(l)): # 0-2 then over the outer ones
newRow.append(l[rowIdx][colIdx])
l2.append(newRow)
print(l2) # [[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
What i meant was something like this:
def t(array): #The original array has size mxn
duplicate = [[0 for x in range(len(array))] for y in range(len(array[1]))] #You create an array of size nxm, which is filled with zeros
for i in range(len(array)): #Loop over the rows
for j in range(len(array[i])): #Then loop over the columns
duplicate[j][i] = array[i][j] #Replace j,i or duplicate with i,j th element of original
return duplicate
Now,
>>> t([[ 1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]