According to the documentation of scipy.ndimage.map_coordinates,
The array of coordinates is used to find, for each point in the
output, the corresponding coordinates in the input. The value of the
input at those coordinates is determined by spline interpolation of
the requested order.
The shape of the output is derived from that of the coordinate array
by dropping the first axis. The values of the array along the first
axis are the coordinates in the input array at which the output value
is found.
I have a discrete 3-d function that is defined on a 3d grid (t, x, y); on every point of this 3d grid, the function has a unique value unless it's value is zero.
I have another set of arrays in the form of a pandas dataframe with three columns, t_new, x_new, and y_new.
I would like to use scipy.ndimage.map_coordinates to interpolate the function in order to calculate its value on the new dataset presented in the said dataframe.
Since I am getting the following error message, I am sure I am not setting up the map_coordinates correctly:
File "D:\Users\username\Anaconda3\lib\site-packages\scipy\ndimage\interpolation.py", line 437, in map_coordinates
raise RuntimeError('invalid shape for coordinate array')
Here is my definition of the interpolation function:
from scipy.ndimage import map_coordinates
def interpolator_3d(df, func_values):
# The coordinates at which input is evaluated
coordinates = df[['t_new', 'x_new', 'y_new']].values.T # (3, 1273)
# list of input array [[t0, x0, y0, value0], [t1, x1, y1, value1], ...]
input_arr = func_values # (1780020000, 4)
return map_coordinates(input_arr, coordinates)
There are at least two issues with how you are using map_coordinates. Keep in mind that this function was designed for image resampling.
If you have a 3d-function the array input_arr should be 3-dimensional. map_coordinates will use the indices as t, x and y coordinates. The value v of the function has to be stored at each respective position. If your original function has another base grid, then you have to normalize everything accordingly to the arrays indices before and after. This requires an equidistant grid as input.
The coordinates have to be an array e.g. of the form [[t_new_0, t_new_1, ...], [x_new_0, x_new_1 ...], [y_new_0, y_new_1, ...]]. The result will be a list of interpolated samples [[v_new_0, v_new_1, ...]]. Generally, if input_array is n-dimensional, coordinates has to be a list that contains n arrays of same shape S. The result will be a list of arrays of shape S.
Example with n=3 dimensions and 5 samples to interpolate in a 1-dimensional shape:
import numpy as np
from scipy import ndimage
a = np.arange(64.).reshape((4, 4, 4))
print(a)
out = ndimage.map_coordinates(a, [
[0.5, 1.0, 1.5, 2.0, 2.5], [0.1, 0.2, 0.3, 0.4, 0.5], [2.0, 1.9, 1.8, 1.7, 1.6]
])
print(out)
Output:
[[[ 0. 1. 2. 3.]
[ 4. 5. 6. 7.]
[ 8. 9. 10. 11.]
[12. 13. 14. 15.]]
[[16. 17. 18. 19.]
[20. 21. 22. 23.]
[24. 25. 26. 27.]
[28. 29. 30. 31.]]
[[32. 33. 34. 35.]
[36. 37. 38. 39.]
[40. 41. 42. 43.]
[44. 45. 46. 47.]]
[[48. 49. 50. 51.]
[52. 53. 54. 55.]
[56. 57. 58. 59.]
[60. 61. 62. 63.]]]
[ 7.6688, 18.148 , 26.3424, 34.6304, 45.3904]
Update:
That means, if your input_array has the form [[t0, x0, y0, value0], [t1, x1, y1, value1], ...] with length 1780020000 = 19778 * 500 * 180 it has to be transformed accordingly to an array of shape (19778, 500, 180):
t_max, x_max, y_max, _ = np.max(func_values, axis=0).astype(int) + 1 # 19778, 500, 180
input_arr = np.zeros((t_max, x_max, y_max), dtype=float)
for t, x, y, v in func_values:
input_arr[int(t), int(x), int(y)] = v
Related
I read this question but it doesnt seem to answer my question :(.
So basically I'm trying to vectorize the game snake so it can run faster.
Here is my code till now:
import torch
import torch.nn.functional as F
device = torch.device("cpu")
class SnakeBoard:
def __init__(self, board=None):
if board != None:
self.channels = board
else:
# 0 - Food, 1 - Head, 2 - Body
self.channels = torch.zeros(1, 3, 15, 17,
device=device)
# Initialize game channels
self.channels[:, 0, 7, 12] = 1
self.channels[:, 1, 7, 5] = 1
self.channels[:, 2, 7, 2:6] = torch.arange(1, 5)
self.move()
def move(self):
self.channels[:, 2] -= 1
F.relu(self.channels[:, 2], inplace=True)
# Up movement test
F.conv2d(self.channels[:, 1], torch.tensor([[[0,1,0],[0,0,0],[0,0,0]]]), padding=1)
SnakeBoard()
The first dimension in channels represents batch size, second dimension represent the 3 channels of the snake game: food, head, and body, and finally the third and fourth dimensions represent the height and width of the board.
Unfortunately when running the code I get error: Expected stride to be a single integer value or a list of 1 values to match the convolution dimensions, but got stride=[1, 1]
How can I fix that?
The dimensions of the inputs for the convolution are not correct for a 2D convolution. Let's have a look at the dimensions you're passing to F.conv2d:
self.channels[:, 1].size()
# => torch.Size([1, 15, 17])
torch.tensor([[[0,1,0],[0,0,0],[0,0,0]]]).size()
# => torch.Size([1, 3, 3])
The correct sizes should be
input: (batch_size, in_channels , height, width)
weight: (out_channels, in_channels , kernel_height, kernel_width)
Because your weight has only 3 dimensions, it is considered to be a 1D convolution, but since you called F.conv2d the stride and padding will be tuples and therefore it won't work.
For the input you indexed the second dimension, which selects that particular element across that dimensions and eliminates that dimensions. To keep that dimension you can index it with a slice of just one element.
And for the weight you are missing one dimension as well, which can just be added directly. Also your weight is of type torch.long, since you are only using integers in the tensor creation, but the weight needs to be of type torch.float.
F.conv2d(self.channels[:, 1:2], torch.tensor([[[[0,1,0],[0,0,0],[0,0,0]]]], dtype=torch.float), padding=1)
On a different note, I don't think that convolutions are appropriate for this use case, because you're not using a key property of the convolution, which is to capture the surroundings. Those are just too many unnecessary computations to achieve what you want, most of them are multiplications with 0.
For example, a move up is much easier to achieve by removing the first row and adding a new row of zeros at the end, so everything is shifted up (assuming that the first row is the top and the last row is the bottom of the board).
head = self.channels[:, 1:2]
batch_size, channels, height, width = head.size()
# Take everything but the first row of the head
# Add a row of zeros to the end by concatenating them across the height (dimension 2)
new_head = torch.cat([head[:, :, 1:], torch.zeros(batch_size, channels, 1, width)], dim=2)
# Or if you want to wrap it around the board, it's even simpler.
# Move the first row to the end
wrap_around_head = torch.cat([head[:, :, 1:], head[:, :, 0:1]], dim=2)
I am trying to train a model where one set of data contain a particular pixDim, whereas another set contains a different pixDim. I want to normalize both the voxel resolution and execute.
Can we change the pixDim dimension of a volumetric data like .nifti.gz or .mgz file using nibabel or any other python library?
For reference, I am talking about pixDim in the header of a volumetric file highlighted in the below image.
The preferable way is to calculate target pixdim or use scipy interpolation method as in below function to achieve target pixdim or steps(in func)
import scipy.interpolate as si
def do_interpolate(values, steps, isLabel=False):
x, y, z = [steps[k] * np.arange(values.shape[k]) for k in range(3)] # original grid
if isLabel:
method = 'nearest'
else:
method = 'linear'
f = si.RegularGridInterpolator((x, y, z), values, method=method) # interpolator
dx, dy, dz = 2.0, 2.0, 3.0 # new step sizes # settings['EVAL']['target_voxel_dimension']
new_grid = np.mgrid[0:x[-1]:dx, 0:y[-1]:dy, 0:z[-1]:dz] # new grid
new_grid = np.moveaxis(new_grid, (0, 1, 2, 3), (3, 0, 1, 2)) # reorder axes for evaluation
return f(new_grid)
You will get an updated upsampled or downsampled resolution for your volume data with target pixdim maintained.
NOTE: In the above function, values hold 3d volumetric data, steps hold original pixdim data, I have hardcoded target pixdim in the form of dx, dy, dz.
I have data which I have applied a cubic spline using interp1d (assuming x and y have been previously defined)
f2 = interp1d(x, y, kind='cubic')
xnew = np.linspace(start, end, num=501, endpoint=True)
spline = np.array(f2(xnew))
if I check this I get the values I am expecting
print(spline)
[ 2.737e-02 5.0504e-02 ....]
However, I need to identify peaks in the spline and am using find_peaks from Numpy Signal
peaks = signal.find_peaks(spline)
But this returns a series of index positions whereas I am trying to return the actual peak values of the spline array.
print(peaks)
(array([ 15, 123, 290, 432]), {})
Reading the documents (https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.find_peaks.html)
this doesn't appear to be the expected behaviour (although it is entirely possible I have misunderstood). Any suggestions?
For the pyplot.scatter(x,y,s,c....) function ,
The matplotlib docs states that :
c : color, sequence, or sequence of color, optional, default: 'b' The
marker color. Possible values:
A single color format string. A sequence of color specifications of
length n. A sequence of n numbers to be mapped to colors using cmap
and norm. A 2-D array in which the rows are RGB or RGBA. Note that c
should not be a single numeric RGB or RGBA sequence because that is
indistinguishable from an array of values to be colormapped. If you
want to specify the same RGB or RGBA value for all points, use a 2-D
array with a single row.
However i do not understand how i can change the colors of the datapoints as i wish .
I have this piece of code :
import matplotlib.pyplot as plt
import numpy as np
import sklearn
import sklearn.datasets
import sklearn.linear_model
import matplotlib
%matplotlib inline
matplotlib.rcParams['figure.figsize'] = (13.0, 9.0)
# Generate a dataset and plot it
np.random.seed(0)
X, y = sklearn.datasets.make_moons(200, noise=0.55)
print(y)
plt.scatter(X[:,0], X[:,1], c=y)#, cmap=plt.cm.Spectral)
the output plot
How can i change the colours to suppose black and green datapoints if i wish ? or something else ? Also please explain what exactly cmap does .
Why my plots are magenta and blue every time i use plt.cm.Spectral ?
There are essentially two option on how to colorize scatter points.
1. External mapping
You may externally map values to color and supply a list/array of those colors to the scatter's c argument.
z = np.array([1,0,1,0,1])
colors = np.array(["black", "green"])
plt.scatter(x,y, c=colors[z])
2. Internal mapping
Apart from explicit colors, one can also supply a list/array of values which should be mapped to colors according to a normalization and a colormap.
A colormap is a callable that takes float values between 0. and 1. as input and returns a RGB color.
A normalization is a callable that takes any number as input and outputs another number, based on some previously set limits. The usual case of Normalize would provide a linear mapping of values between vmin and vmax to the range between 0. and 1..
The natural way to obtain a color from some data is hence to chain the two,
cmap = plt.cm.Spectral
norm = plt.Normalize(vmin=4, vmax=5)
z = np.array([4,4,5,4,5])
plt.scatter(x,y, c = cmap(norm(z)))
Here the value of 4 would be mapped to 0 by the normalzation, and the value of 5 be mapped to 1, such that the colormap provides the two outmost colors.
This process happens internally in scatter if an array of numeric values is provided to c.
A scatter creates a PathCollection, which subclasses ScalarMappable. A ScalarMappable consists of a colormap, a normalization and an array of values. Hence the above is internalized via
plt.scatter(x,y, c=z, norm=norm, cmap=cmap)
If the minimum and maximum data are to be used as limits for the normalization, you may leave that argument out.
plt.scatter(x,y, c=z, cmap=cmap)
This is the reason that the output in the question will always be purple and yellow dots, independent of the values provided to c.
Coming back to the requirement of mapping an array of 0 and 1 to black and green color you may now look at the colormaps provided by matplotlib and look for a colormap which comprises black and green. E.g. the nipy_spectral colormap
Here black is at the start of the colormap and green somewhere in the middle, say at 0.5. One would hence need to set vmin to 0, and vmax, such that vmax*0.5 = 1 (with 1 the value to be mapped to green), i.e. vmax = 1./0.5 == 2.
import matplotlib.pyplot as plt
import numpy as np
x,y = np.random.rand(2,6)
z = np.array([0,0,1,1,0,1])
plt.scatter(x,y, c = z,
norm = plt.Normalize(vmin=0, vmax=2),
cmap = "nipy_spectral")
plt.show()
Since there may not always be a colormap with the desired colors available and since it may not be straight forward to obtain the color positions from existing colormaps, an alternative is to create a new colormaps specifically for the desired purpose.
Here we might simply create a colormap of two colors black and green.
matplotlib.colors.ListedColormap(["black", "green"])
We would not need any normalization here, because we only have two values and can hence rely on automatic normalization.
import matplotlib.pyplot as plt
import matplotlib.colors as mcolors
import numpy as np
x,y = np.random.rand(2,6)
z = np.array([0,0,1,1,0,1])
plt.scatter(x,y, c = z, cmap = mcolors.ListedColormap(["black", "green"]))
plt.show()
First, to set the colors according to the values in y, you can do this:
color = ['red' if i==0 else 'green' for i in y]
plt.scatter(X[:,0], X[:,1], c=color)
Now talking about scatter() and cmap.
ColorMaps are used to provide colors from float values. See this documentation for reference on colormaps.
For values between 0 to 1, a color is chosen from these colormaps.
For example:
plt.cm.Spectral(0.0)
# (0.6196078431372549, 0.00392156862745098, 0.25882352941176473, 1.0) #<== magenta
plt.cm.Spectral(1.0)
# (0.3686274509803922, 0.30980392156862746, 0.6352941176470588, 1.0) #<== blue
plt.cm.Spectral(1)
# (0.6280661284121491, 0.013302575932333718, 0.26082276047673975, 1.0)
Note that the results of 1.0 and 1 are different in above code, because the int and floats are handled differently as mentioned in documentation of __call__() here:
For floats, X should be in the interval [0.0, 1.0] to return the
RGBA values X*100 percent along the Colormap line.
For integers, X should be in the interval [0, Colormap.N) to
return RGBA values indexed from the Colormap with index X.
Please look at this answer for more better explanation about colormaps:-
https://stackoverflow.com/a/25408562/3374996
In your y, you have 0 and 1, so the RGBA values shown in above code are used (which are representing two ends of the Spectral colormap).
Now here's how c and cmap parameters in plt.scatter() interact with each other.
_______________________________________________________________________
|No | type of x, y | c type | values in c | result |
|___|______________|__________|_____________|___________________________|
|1 | single | scalar | numbers | cmap(0.0), no matter |
| | point | | | what the value in c |
|___|______________|__________|_____________|___________________________|
|2 | array of | array | numbers | normalize the values in c,|
| | points | | | cmap(normalized val in c) |
|___|______________|__________|_____________|___________________________|
|3 | scalar or | scalar or| RGBA Values,| no use of cmap, |
| | array | array |Color Strings| use colors from c |
|___|______________|__________|_____________|___________________________|
Now once the actual colors are finalized, then cycles through the colors for each point in x, y. If the size of x, y is equal to or less than size of colors in c, then you get perfect mapping, or else olders colors are used again.
Here's an example to illustrate this:
# Case 1 from above table
# All three points get the same color = plt.cm.Spectral(0)
plt.scatter(x=0.0, y=0.2, c=0, cmap=plt.cm.Spectral)
plt.scatter(x=0.0, y=0.3, c=1, cmap=plt.cm.Spectral)
plt.scatter(x=0.0, y=0.4, c=1.0, cmap=plt.cm.Spectral)
# Case 2 from above table
# The values in c are normalized
# highest value in c gets plt.cm.Spectral(1.0)
# lowest value in c gets plt.cm.Spectral(0.0)
# Others in between as per normalizing
# Size of arrays in x, y, and c must match here, else error is thrown
plt.scatter([0.1, 0.1, 0.1, 0.1, 0.1], [0.2, 0.3, 0.4, 0.5, 0.6],
c=[1, 2, 3, 4, 5], cmap=plt.cm.Spectral)
# Case 3 from above table => No use of cmap here,
# blue is assigned to the point
plt.scatter(x=0.2, y=0.3, c='b')
# You can also provide rgba tuple
plt.scatter(x=0.2, y=0.4, c=plt.cm.Spectral(0.0))
# Since a single point is present, the first color (green) is given
plt.scatter(x=0.2, y=0.5, c=['g', 'r'])
# Same color 'cyan' is assigned to all values
plt.scatter([0.3, 0.3, 0.3, 0.3, 0.3], [0.2, 0.3, 0.4, 0.5, 0.6],
c='c')
# Colors are cycled through points
# 4th point will get again first color
plt.scatter([0.4, 0.4, 0.4, 0.4, 0.4], [0.2, 0.3, 0.4, 0.5, 0.6],
c=['m', 'y', 'k'])
# Same way for rgba values
# Third point will get first color again
plt.scatter([0.5, 0.5, 0.5, 0.5, 0.5], [0.2, 0.3, 0.4, 0.5, 0.6],
c=[plt.cm.Spectral(0.0), plt.cm.Spectral(1.0)])
Output:
Go through the comments in the code and location of points along with the colors to understand thoroughly.
You can also replace the param c with color in the code of Case 3 and the results will still be same.
I am unable to clearly comprehend theano's reshape. I have an image matrix of shape:
[batch_size, stack1_size, stack2_size, height, width]
, where there are stack2_size stacks of images, each having stack1_size of channels. I now want to convert them into the following shape:
[batch_size, stack1_size*stack2_size, 1 , height, width]
such that all the stacks will be combined together into one stack of all channels. I am not sure if reshape will do this for me. I see that reshape seems to not lexicographically order the pixels if they are mixed in dimensions in the middle. I have been trying to achieve this with a combination of dimshuffle,reshape and concatenate, but to no avail. I would appreciate some help.
Thanks.
Theano reshape works just like numpy reshape with its default order, i.e. 'C':
‘C’ means to read / write the elements using C-like index order, with
the last axis index changing fastest, back to the first axis index
changing slowest.
Here's an example showing that the image pixels remain in the same order after a reshape via either numpy or Theano.
import numpy
import theano
import theano.tensor
def main():
batch_size = 2
stack1_size = 3
stack2_size = 4
height = 5
width = 6
data = numpy.arange(batch_size * stack1_size * stack2_size * height * width).reshape(
(batch_size, stack1_size, stack2_size, height, width))
reshaped_data = data.reshape([batch_size, stack1_size * stack2_size, 1, height, width])
print data[0, 0, 0]
print reshaped_data[0, 0, 0]
x = theano.tensor.TensorType('int64', (False,) * 5)()
reshaped_x = x.reshape((x.shape[0], x.shape[1] * x.shape[2], 1, x.shape[3], x.shape[4]))
f = theano.function(inputs=[x], outputs=reshaped_x)
print f(data)[0, 0, 0]
main()