I have a Keras network with two inputs:
image of shape (128, 128, 3)
bounding-box of shape (4), i.e. (x0, y0, x1, y1)
In my network definition, I need to include the extraction of the image patch defined by the bounding-box from the input image, but I do not know how (or my attempts did not work). Here is my current attempt to achieve this, can someone please help me to understand slicing Tensors by Values of other Tensors in Keras?
# get masked image and bounding box information as inputs
masked_img = Input(shape=self.input_shape)
mask_bounding_box = Input(shape=(4,))
# fill in the masked region and extract the fill-in region
filled_img = self.generator(masked_img)
fill_in = K.slice(filled_img, (int(mask_bounding_box[0]), int(mask_bounding_box[1])),
(int(mask_bounding_box[2]), int(mask_bounding_box[3])))
Does anybody know how to do this? Any hint in the right direction would help me, please ...
Thanks in advance!
here's a native numpy solution.
import numpy as np
a = np.arange(48).reshape(3,4,4)
a
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23],
[24, 25, 26, 27],
[28, 29, 30, 31]],
[[32, 33, 34, 35],
[36, 37, 38, 39],
[40, 41, 42, 43],
[44, 45, 46, 47]]])
box = (1,1,2,2) # slicing from (1,1) to (2,2)
b = a[:, box[0]:box[2]+1, box[1]:box[3]+1] # slicing on all channels
b
array([[[ 5, 6],
[ 9, 10]],
[[21, 22],
[25, 26]],
[[37, 38],
[41, 42]]])
Keras.backend.slice() requires starts and offsets, so you could do it like this:
import keras.backend as K
start=(0,1,1) # 1st channel, x1, y1
sizes=(3,2,2) # number of channels, x2-x1+1, y2-y1+1
with sess.as_default():
b=K.slice(a, start, sizes)
print(b.eval())
[[[ 5 6]
[ 9 10]]
[[21 22]
[25 26]]
[[37 38]
[41 42]]]
Related
How can I slice a 3D tensor using a 1D tensor? For instance, consider the following 2 tensors: t of size [Batch, Sequence, Dim]; and idx of size [Batch]. The values of idx are restricted to be integers between 0 and Sequence-1.
I need tensor idx to select the corresponding slices in the second dimension of tensor t. For example:
t = torch.arange(24).view(2,3,4)
>>> tensor([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
idx = torch.tensor([0,2])
>>> tensor([0, 2])
Then the desired output is: tensor([[ 0, 1, 2, 3], [20, 21, 22, 23]]).
The following code solves the problem, however it's inefficient, as it involves one_hot, multiplication and sum operations.
one_hot_idx = nn.functional.one_hot(idx.long(), num_classes=t.shape[1]).unsqueeze(-1)
(t*one_hot_idx).sum(1)
You can do it like this:
import torch
t = torch.arange(24).view(2, 3, 4)
idx = torch.tensor([0, 2])
print(t[range(len(idx)), idx])
Output:
tensor([[ 0, 1, 2, 3],
[20, 21, 22, 23]])
There has a 3-dimensional array x of shape (2000,60,5). If we think it represents a video, the 2000 can represent 2000 frames. I would like to randomly sample it along with the first dimension, i.e., get a set of frame samples. For instance, how to get an array of (500,60,5) which is randomly sampled from x along with the first dimension?
You can pass x as the first argument of the choice method. If you don't want repeated frames in your sample, use replace=False.
For example,
In [10]: x = np.arange(72).reshape(9, 2, 4) # Small array for the demo.
In [11]: x
Out[11]:
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[16, 17, 18, 19],
[20, 21, 22, 23]],
[[24, 25, 26, 27],
[28, 29, 30, 31]],
[[32, 33, 34, 35],
[36, 37, 38, 39]],
[[40, 41, 42, 43],
[44, 45, 46, 47]],
[[48, 49, 50, 51],
[52, 53, 54, 55]],
[[56, 57, 58, 59],
[60, 61, 62, 63]],
[[64, 65, 66, 67],
[68, 69, 70, 71]]])
Sample "frames" from x with the choice method of NumPy random generator instance.
In [12]: rng = np.random.default_rng()
In [13]: rng.choice(x, size=3)
Out[13]:
array([[[40, 41, 42, 43],
[44, 45, 46, 47]],
[[40, 41, 42, 43],
[44, 45, 46, 47]],
[[16, 17, 18, 19],
[20, 21, 22, 23]]])
In [14]: rng.choice(x, size=3, replace=False)
Out[14]:
array([[[ 8, 9, 10, 11],
[12, 13, 14, 15]],
[[32, 33, 34, 35],
[36, 37, 38, 39]],
[[ 0, 1, 2, 3],
[ 4, 5, 6, 7]]])
Note that the frames will be in random order; if you want to preserve the order, you could use choice to generate an array of indices, then use the sorted indices to pull the frames out of x.
There exist an numpy ndarry A of shape [100,50, 5], and I want to expand A as follows. A will be appended with an one-dimensional array of shape (50, ). The resulting A will have shape [100,50,6].
The element of this one-dimensional array is based on the array in the original ndarray, i.e., A[:,:,4] in terms of a given formula, i.e., A[:,i,5]=A[:,i,4]*B[i]+5 for i = 0:49 Here A[:,:,5] corresponds to the added one-dimensional array. B is another array working as weight.
Besides using a for loop to write this function, how to fullfill this task in a vectorized/efficient way leveraging numpy operation
Make 2 arrays - with sizes that we can look at:
In [371]: A = np.arange(24).reshape(2,3,4); B = np.array([10,20,30])
Due to broadcasting we can add a (3,) array to (2,3) array
In [372]: A[:,:,-1]+B
Out[372]:
array([[13, 27, 41],
[25, 39, 53]])
we can then convert that to (2,3,1) array:
In [373]: (A[:,:,-1]+B)[:,:,None]
Out[373]:
array([[[13],
[27],
[41]],
[[25],
[39],
[53]]])
In [374]: A
Out[374]:
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
and join them on the last axis:
In [375]: np.concatenate((A, Out[373]), axis=-1)
Out[375]:
array([[[ 0, 1, 2, 3, 13],
[ 4, 5, 6, 7, 27],
[ 8, 9, 10, 11, 41]],
[[12, 13, 14, 15, 25],
[16, 17, 18, 19, 39],
[20, 21, 22, 23, 53]]])
Or we can make a target array of the right size, and copy values to it:
In [376]: A1 = np.zeros((2,3,5),int)
In [377]: A1[:,:,:-1]=A
In [379]: A1[:,:,-1]=Out[372]
With a 3D tensor of shape (number of filters, height, width), how can one reduce the number of filters with a reshape which keeps the original filters together as whole blocks?
Assume the new size has dimensions chosen such that a whole number of the original filters can fit side by side in one of the new filters. So an original size of (4, 2, 2) can be reshaped to (2, 2, 4).
A visual explanation of the side by side reshape where you see the standard reshape will alter the individual filter shapes:
I have tried various pytorch functions such as gather and select_index but not found a way to get to the end result in a general manner (i.e. works for different numbers of filters and different filter sizes).
I think it would be easier to rearrange the tensor values after performing the reshape but could not get a tensor of the pytorch reshaped form:
[[[1,2,3,4],
[5,6,7,8]],
[[9,10,11,12],
[13,14,15,16]]]
to:
[[[1,2,5,6],
[3,4,7,8]],
[[9,10,13,14],
[11,12,15,16]]]
for completeness, the original tensor before reshaping:
[[[1,2],
[3,4]],
[[5,6],
[7,8]],
[[9,10],
[11,12]],
[[13,14],
[15,16]]]
Another option is to construct a list of parts and concatenate them
x = torch.arange(4).reshape(4, 1, 1).repeat(1, 2, 2)
y = torch.cat([x[i::2] for i in range(2)], dim=2)
print('Before\n', x)
print('After\n', y)
which gives
Before
tensor([[[0, 0],
[0, 0]],
[[1, 1],
[1, 1]],
[[2, 2],
[2, 2]],
[[3, 3],
[3, 3]]])
After
tensor([[[0, 0, 1, 1],
[0, 0, 1, 1]],
[[2, 2, 3, 3],
[2, 2, 3, 3]]])
Or a little more generally we could write a function that takes groups of neighbors along a source dimension and concatenates them along a destination dimension
def group_neighbors(x, group_size, src_dim, dst_dim):
assert x.shape[src_dim] % group_size == 0
return torch.cat([x[[slice(None)] * (src_dim) + [slice(i, None, group_size)] + [slice(None)] * (len(x.shape) - (src_dim + 2))] for i in range(group_size)], dim=dst_dim)
x = torch.arange(4).reshape(4, 1, 1).repeat(1, 2, 2)
# read as "take neighbors in groups of 2 from dimension 0 and concatenate them in dimension 2"
y = group_neighbors(x, group_size=2, src_dim=0, dst_dim=2)
print('Before\n', x)
print('After\n', y)
You could do it by chunking tensor and then recombining.
def side_by_side_reshape(x):
n_pairs = x.shape[0] // 2
filter_size = x.shape[-1]
x = x.reshape((n_pairs, 2, filter_size, filter_size))
return torch.stack(list(map(lambda x: torch.hstack(x.unbind()), k)))
>> p = torch.arange(1, 91).reshape((10, 3, 3))
>> side_by_side_reshape(p)
tensor([[[ 1, 2, 3, 10, 11, 12],
[ 4, 5, 6, 13, 14, 15],
[ 7, 8, 9, 16, 17, 18]],
[[19, 20, 21, 28, 29, 30],
[22, 23, 24, 31, 32, 33],
[25, 26, 27, 34, 35, 36]],
[[37, 38, 39, 46, 47, 48],
[40, 41, 42, 49, 50, 51],
[43, 44, 45, 52, 53, 54]],
[[55, 56, 57, 64, 65, 66],
[58, 59, 60, 67, 68, 69],
[61, 62, 63, 70, 71, 72]],
[[73, 74, 75, 82, 83, 84],
[76, 77, 78, 85, 86, 87],
[79, 80, 81, 88, 89, 90]]])
but I know it's not ideal since there is map, list and unbind which disrupts memory. This is what I offer till I figure out how to do it via view only (so a real reshape)
I have an array off this form:
data = [[19, 14, 6, 36, 3],
[12, 12, 1, 32, 1],
[18, 25, 0, 33, 0],
[13, 19, 0, 32, 5],
[12, 14, 0, 33, 0],
[16, 14, 7, 30, 0],
[11, 18, 5, 31, 2],
[17, 11, 3, 46, 7]]
I want to plot it as a bar chart. There would be 8 points on the x-axis, each having 5 bars, with heights corresponding to the 5 values in each row of the array. Would super appreciate any help!
There are two obtions using plt.bar.
Single, adjacent bars
You can either plot the bars next to each other, in a grouped fashion, where you need to determine the bars' positions from the number of columns in the array.
import numpy as np
import matplotlib.pyplot as plt
data = np.array([[19, 14, 6, 36, 3],
[12, 12, 1, 32, 1],
[18, 25, 0, 33, 0],
[13, 19, 0, 32, 5],
[12, 14, 0, 33, 0],
[16, 14, 7, 30, 0],
[11, 18, 5, 31, 2],
[17, 11, 3, 46, 7]])
x = np.arange(data.shape[0])
dx = (np.arange(data.shape[1])-data.shape[1]/2.)/(data.shape[1]+2.)
d = 1./(data.shape[1]+2.)
fig, ax=plt.subplots()
for i in range(data.shape[1]):
ax.bar(x+dx[i],data[:,i], width=d, label="label {}".format(i))
plt.legend(framealpha=1).draggable()
plt.show()
Stacked bars
Or you can stack the bars on top of each other, such that the bottom of the bar starts at the top of the previous one.
import numpy as np
import matplotlib.pyplot as plt
data = np.array([[19, 14, 6, 36, 3],
[12, 12, 1, 32, 1],
[18, 25, 0, 33, 0],
[13, 19, 0, 32, 5],
[12, 14, 0, 33, 0],
[16, 14, 7, 30, 0],
[11, 18, 5, 31, 2],
[17, 11, 3, 46, 7]])
x = np.arange(data.shape[0])
fig, ax=plt.subplots()
for i in range(data.shape[1]):
bottom=np.sum(data[:,0:i], axis=1)
ax.bar(x,data[:,i], bottom=bottom, label="label {}".format(i))
plt.legend(framealpha=1).draggable()
plt.show()