I try to concatenate two numpy arrays with 3 dimensions.
I Actually tried with np.append and np.concatenate.
The dimensions of what I want to concatenate are
((10, 61, 1) (1, 0, 0)).
So I want to save the same structure. The final results is something with this size ( ,) So Python put everything on the same dimension
matrix_results_final = np.empty((1,0,0), float)
vec_y_labels_final = np.empty((1), int)
dict_minmax_scall = searchMinMaxValuesRawDataByElectrode(raw_data)
for electrode_number in electrode_numbers:
#... Some code
matrix_results_final = np.append(matrix_results_final, matrix_results)
Where the matrix_results_final is (1,0,0) and the matrix_results (10,61,1)
So I want to wave (matrix_results_final (10,61,1) in the first iteration, (20,61,1) on the second,...
I don't have any error message.
if i understood you correctly you want to append some data in a loop like:
import numpy as np
arr = np.random.rand(10,61,1)
for i in range(3):
print(arr.shape)
arr2 = np.random.rand(10,61,1)
arr = np.concatenate([arr,arr2],axis=0)
print(arr.shape)
(10, 61, 1)
(20, 61, 1)
(30, 61, 1)
(40, 61, 1)
note that it would be more effective to first assign a empty array with the final shape and then fill it (so we don't have to find new memory space - because the array doesn't change the size - in each iteration)
arr = np.zeros((40,61,1))
for i in range(4):
ii = i * 10
arr[ii:ii+10,:,:] = np.random.rand(10,61,1)
Related
I have a 2d array of the shape (25, 3) where the first dimension correspond to position and the second dimension correspond to file number.
I would like to assign a 2d array of the shape (100000, 13) to every position of every file where the first dimension is time and the second dimension is some physical quantity. What produces this 2d array is the output of solving a DAE for every given position of every file.
The output should be of the shape (25, 3, 100000, 13) where there is a solution for every position.
Here is my attempt:
import numpy as np
from scipy.integrate import solve_ivp
def integrator(soln, finter, finter_val, finter_alpha, finter_beta, i: int, num_files: int):
"""This integrator is a function of few lists, finter, finter_val, finter_alpha,
and finter_beta where each list is composed of "InterpolatedUnivariateSpline" objects"""
args = (finter[i][num_files], finter_val[i][num_files], finter_alpha[i][num_files],
finter_beta[i][num_files])
# eqn : some complicated function
# t_span : Interval of integration (t0, tf)
# y0: Initial state
# t_eval: Times at which to store the computed solution
sol = solve_ivp(
eqns,
t_span=t_span,
y0=y0,
method='BDF',
#dense_output=True,
t_eval=t,
vectorized=True,
args=args)
soln[i][num_files] = sol.y.T
def ode_solver_serial(finter, finter_val, finter_alpha, soln, finter_beta):
soln = np.zeros(shape=(25, 3), dtype=np.float64).tolist()
print('Before', np.array(soln, dtype=object).shape) # should be (25, 3)
for fnum in range(3):
for i in range(25):
integrator(soln, finter, finter_val, finter_alpha, finter_beta, i, fnum)
print('After', np.array(soln, dtype=object).shape) # should be (25, 3, 100000, 13)
Here is the output I am getting which is not right:
Before (25, 3)
After (25, 3) #this should be (25, 3, 100000, 13)
I have a tensor T with dimension (d1 x d2 x d3 x ... dk) and a tensor I with dimension (p x q). Here, I contains coordinates of T but q < k, each column of I corresponds to a dimension of T. I have another tensor V of dimension p x di x ...dj where sum([di, ..., dj]) = k - q. (di, .., dj) corresponds to missing dimensions from I. I need to perform T[I] = V
A specific example of such problem using numpy array posted here[1].
The solution[2] uses fancy indexing[3] which relies on numpy.index_exp. In case of pytorch such option is not available. Is there any alternative way to mimic this in pytorch without using loops or casting tensors to numpy array?
Below is a demo:
import torch
t = torch.randn((32, 16, 60, 64)) # tensor
i0 = torch.randint(0, 32, (10, 1)).to(dtype=torch.long) # indexes for dim=0
i2 = torch.randint(0, 60, (10, 1)).to(dtype=torch.long) # indexes for dim=2
i = torch.cat((i0, i2), 1) # indexes
v = torch.randn((10, 16, 64)) # to be assigned
# t[i0, :, i2, :] = v ?? Obviously this does not work
[1] Slice numpy array using list of coordinates
[2] https://stackoverflow.com/a/42538465/6422069
[3] https://numpy.org/doc/stable/reference/generated/numpy.s_.html
After some discussion in the comments, we arrived at the following solution:
import torch
t = torch.randn((32, 16, 60, 64)) # tensor
# indices
i0 = torch.randint(0, 32, (10,)).to(dtype=torch.long) # indexes for dim=0
i2 = torch.randint(0, 60, (10,)).to(dtype=torch.long) # indexes for dim=2
v = torch.randn((10, 16, 64)) # to be assigned
t[(i0, slice(None), i2, slice(None))] = v
Hi i tried to write a code, where i write a number on the screen with pygame and then a neural Network predicts the number i wrote. My problem is that i trained my neural network with image arrays in a (28, 28, 3). So i tried to reshape my (280, 280, 3) array. but when i do so my array is None.
I use Python 3.7
string_image = pygame.image.tostring(screen, 'RGB')
temp_surf = pygame.image.fromstring(string_image, (280, 280), 'RGB')
array = pygame.surfarray.array3d(temp_surf)
array = array.resize((28, 28, 3))
Can anyone help?
If you just want to scale a pygame.Surface, then I recommend to use pygame.transform.scale() or pygame.transform.smoothscale().
For instance:
temp_surf = pygame.image.fromstring(string_image, (280, 280), 'RGB')
scaled_surf = pygame.transform.smoothscale(temp_surf, (28, 28))
array = pygame.surfarray.array3d(scaled_surf)
I am not familiar with pygame but it looks like pygame.surfarray.array3d returns a numpy-array.
How to iterate through a pygame 3d surfarray and change the individual color of the pixels if they're less than a specific value?
To keep the same number of data points and just change the shape you can use numpy.reshape.
import numpy as np
c = 28*28*3 # just to start off with right amount of data
a = np.arange(c) # this just creates the initial data you want to format
b = a.reshape((28,28,3)) # this actually reshapes the data
d = b.shape #this just stores the new shape in variable d
print(d) #shows you the new shape
To interpolate the data to a new size then there are a number of ways like CV or continuing a numpy example.
ary = np.resize(b,(18,18,1))
e = ary.shape
print(e)
Hope that helps.
I'm trying to use dask.array.map_blocks to process a dask array, using a second dask array with different shape as an argument. The use case is firstly running some peak finding on a 2-D stack of images (4-dimensions), which is returned as a 2-D dask array of np.objects. Ergo, the two first dimensions of the two dask arrays are the same. The peaks are then used to extract intensities from the 4-dimensional dataset. In the code below, I've omitted the peak finding part. Dask version 1.0.0.
import numpy as np
import dask.array as da
def test_processing(data_chunk, position_chunk):
output_array = np.empty(data_chunk.shape[:-2], dtype='object')
for index in np.ndindex(data_chunk.shape[:-2]):
islice = np.s_[index]
intensity_list = []
data = data_chunk[islice]
positions = position_chunk[islice]
for x, y in positions:
intensity_list.append(data[x, y])
output_array[islice] = np.array(intensity_list)
return output_array
data = da.random.random(size=(4, 4, 10, 10), chunks=(2, 2, 10, 10))
positions = np.empty(data.shape[:-2], dtype='object')
for index in np.ndindex(positions.shape):
positions[index] = np.arange(10).reshape(5, 2)
data_output = da.map_blocks(test_processing, data, positions, dtype=np.object,
chunks=(2, 2), drop_axis=(2, 3))
data_output.compute()
This gives the error ValueError: Can't drop an axis with more than 1 block. Please useatopinstead., which I'm guessing is due to positions having 3 dimensions, while data has 4 dimensions.
The same function, but without the positions dask array works fine.
import numpy as np
import dask.array as da
def test_processing(data_chunk):
output_array = np.empty(data_chunk.shape[:-2], dtype='object')
for index in np.ndindex(data_chunk.shape[:-2]):
islice = np.s_[index]
intensity_list = []
data = data_chunk[islice]
positions = [[5, 2], [1, 3]]
for x, y in positions:
intensity_list.append(data[x, y])
output_array[islice] = np.array(intensity_list)
return output_array
data = da.random.random(size=(4, 4, 10, 10), chunks=(2, 2, 10, 10))
data_output = da.map_blocks(test_processing, data, dtype=np.object,
chunks=(2, 2), drop_axis=(2, 3))
data_computed = data_output.compute()
This has been fixed in more recent versions of dask: running the same code on version 2.3.0 of dask works fine.
This is my code:
import random
import numpy as np
import math
populacao = 5
x_min = -10
x_max = 10
nbin = 4
def fitness(xy, populacao, resultado):
fit = np.matrix(resultado)
xy_fit = np.append(xy, fit.T, axis = 1)
xy_fit_sorted = xy_fit[np.argsort(xy_fit[:,-1].T),:]
return xy_fit_sorted
def codifica(x, x_min, x_max,n):
x = float(x)
xdec = round((x-x_min)/(x_max-x_min)*(2**n-1))
xbin = int(bin(xdec)[2:])
return(xbin)
xy = np.array([[1, 2],[3,4],[0,0],[-5,-1],[9,-2]])
resultado = np.array([5, 25, 0, 26, 85])
print(xy)
xy_fit_sorted = np.array(fitness(xy, populacao, resultado))
print(xy_fit_sorted)
parents = (xy_fit_sorted[:,:2])
print(parents)
the problem i'm having is that to select the 2 rows of "xy_fit_sorted", i'm doing this strange thing:
parents = (xy_fit_sorted[:,:2])
Intead of what makes sense in my mind:
parents = (xy_fit_sorted[:1,:])
it's like the whole matrix is in one line.
I'm not sure what most of your code is doing, so here's just a guess: are you thrown off by the shape of xy_fit_sorted being (1, 5, 3), having an extra zero axis?
That could be fixed e.g. by constructing xy_fit without the use of np.matrix:
xy_fit = np.append(xy, resultado[:, np.newaxis], axis=1)
Then xy_fit_sorted comes out with a shape of (5, 3).
The underlying issue was that np.matrix is always a 2-D array. When indexing xy_fit[...] you intend to index with a vector. But using np.matrix for xy_fit, xy_fit[:,-1].T is not a vector, but a 2-D array as well (of shape (1,5)). This leads to xy_fit_sorted having an extra dimension as well.
Note that the numpy doc says about np.matrix anyhow:
It is no longer recommended to use this class, even for linear algebra. Instead use regular arrays. The class may be removed in the future.