I want to read from a .npy file to do some signal processing tasks but during this task I received this error:
IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices
this is my code:
import numpy as np
import matplotlib.pyplot as plt
file = '/signal/data.npy'
d = np.load(file,allow_pickle=True,encoding = 'latin1')
d['soma'][0]
There are same questions but I could not use them to solve this one.So can anyone help me to fix It?
Thanks
This the error:
This is part of my data( d is equal to res):
your data consists of arrays of dictionaries. for each array you have some keys with its values.
the solution as # hpaulj said is:
res[array_index]["your_key"]
You have a numpy array d and you are trying to access e.g. "soma" index which is not possible. Numpy indexing rule is:
only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices.
If your numpy array includes dictionaries, you need to extract dictionaries. d['soma'] does not extract elements of numpy array.
This loops over array d and extracts the first element of values of key 'soma' for all dictionaries in d that has key 'soma':
lfp = [i['soma'][0] for i in d if 'soma' in i]
And if it is a dataframe instead of numpy array, try:
d = pd.read_pickle(file)
Related
In a for loop, I read iteratively one vector from file that then I want to put in a list or numpy array. I don't really understand how this process works for numpy arrays or lists. Since I know numpy arrays are not done to change size, I wanted to use an empty list and iteratively append the vector I'm reading
import numpy as np
a = np.array([[1],[2],[3]])
b = np.array([[2],[3],[4]])
c = timeStep = list()
c = c.append(a)
c = c.append(b)
The example above describes what I would like to do but, when I print the c list after appending a, the terminal shows there is nothing inside.
If You want to create a list I suggest you to use:
c = [] #this is an empty list in Python
Now, if You want to append a numpy array in that list, You should use:
c.append(YourNumpyArray)
Your code should look like this:
a = np.array([[1],[2],[3]])
b = np.array([[2],[3],[4]])
c = []
c.append(a)
c.append(b)
print(c)
Notice that You don't need to do c = c.append()
As you can see in the figure also, the append doesn't return anything. It appends to the original array c. So you are overwriting the appended array with None.
This should be the working code for you.
import numpy as np
a = np.array([[1],[2],[3]])
b = np.array([[2],[3],[4]])
c = timeStep = list()
c.append(a)
c.append(b)
On a side note, yes you can use python lists with numpy arrays. If the number of elements in the list c is fixed or pre-known, you can make it a numpy array too.
I have a strange situation which I do not understand. I wand to use an array as indices.
batch_index = np.arange(min(mb_size,len(D)), dtype=np.int32)
Q_target[batch_index, actions] = rewards + gamma*np.max(Q_next, axis=1)*dones
But when I run this I get an error:
IndexError: arrays used as indices must be of integer (or boolean)
type
It appears to be an array of objects:
In: actions
Out: array([2, 2, 2, 2], dtype=object)
The same in variable explorer:
Type: Array of object Value: ndarray object of numpy module
In the same time:
In:type(actions[0])
Out: numpy.int64
I know that I can use:
actions = np.array(actions, dtype=np.int32)
But I do not understand why do I have to do it.
P.S. here is how I get actions
D = deque()
D.append((state, action, reward, state_new, done))
#type(action) == int64
minibatch = np.array(random.sample(D, min(mb_size,len(D))))
actions = minibatch[:,1]
In my opinion it's because actions is of mixed type (i.e. done is Boolean, reward is most probably some sort of numeric etc.) because you get it from queue object.
In python in general, and specifically in NumPy, when you have a mixed, non-numeric object types - the array is cast to an Object (or other unifying type, that all items agree on). So in your case, the reason you can't index the data with this array is not due to the batch_index, but because of the actions array.
A basic question about populating np arrays from a list:
m is a numpy array with shape (4,486,9).
d is a list with length 23328 and a varying number of items for each index.
I am iterating through m on dimension 1 and 2 and d on dimension 1.
I want to import 9 "columns" from particular lines of d at constant intervals, into m. 6 of those columns are successive, they are shown below with index "some_index".
What I have done below works okay but looks really heavy in syntax, and just wrong. There must be a way to export the successive columns more efficiently?
import numpy as np
m=np.empty(4,486,9)
d=[] #list filled in from files
#some_index is an integer incremented in the loops following some conditions
#some_other_index is another integer incremented in the loops following some other conditions
For i in something:
For j in another_thing:
m[i][j]=[d[some_index][-7], d[some_index][-6], d[some_index][-5], d[some_index][-4], d[some_index][-3], d[some_index][-2], d[some_other_index][4], d[some_other_index][0], d[some_other_index][4]]
Without much imagination, I tried the followings which do not work as np array needs a coma to differentiate items:
For i in something:
For j in another_thing:
m[i][j]=[d[some_index][-7:-1], d[some_other_index][4], d[some_other_index][0], d[some_other_index][4]]
ValueError: setting an array element with a sequence.
m[i][j]=[np.asarray(d[some_index][-7:-1]), d[some_other_index][4], d[some_other_index][0], d[some_other_index][4]]
ValueError: setting an array element with a sequence.
Thanks for your help.
Is this what you are looking for?
You can make use of numpy arrays to select multiple elements at once.
I have taken the liberty to create some data in order to make sure we are doing the right thing
import numpy as np
m=np.zeros((4,486,9))
d=[[2,1,2,3,1,12545,45,12], [12,56,34,23,23,6,7,4,173,47,32,3,4], [7,12,23,47,24,13,1,2], [145,45,23,45,56,565,23,2,2],
[54,13,65,47,1,45,45,23], [125,46,5,23,2,24,23,5,7]] #list filled in from files
d = np.asarray([np.asarray(i) for i in d]) # this is where the solution lies
something = [2,3]
another_thing = [10,120,200]
some_index = 0
some_other_index = 5
select_elements = [-7,-6,-5,-4,-3,-2,4,0,4] # this is the order in which you are selecting the elements
for i in something:
for j in another_thing:
print('i:{}, j:{}'.format(i, j))
m[i,j,:]=d[some_index][select_elements]
Also, I noticed you were indexing this way m[i][j] = .... You can do the same with m[i,j,:] = ...
This seems trivial (again) but has me stumped.
I need to find the indexes of multiple values in a numpy array. I can do this with where and isin but the resulting answer always has a length of 1 regardless of how many indexes are found. Example
import numpy as np
a = [1,3,5,7,9,11,13,15]
b = [1,7,13]
x = np.where(np.isin(a,b))
print(x)
print(len(x))
this returns
(array([0, 3, 6]),)
1
I think its because the array is a single item inside a tuple. How do I return just the array?
Just use
x = np.where(np.isin(a,b))[0]
to get what you expect.
As hpaulj points out in the comments where returns a tuple with one array for each input array dimension, in this case there is only one, which is why x is a tuple of length one.
I was trying to scan an array for values and take action depending on the result. However, when I had a closer look at what the code was doing I noticed that my logical condition was ill posed.
I will illustrate what I mean with the following example:
#importing numpy
import numpy as np
#creating a test array
a = np.zeros((3,3))
#searching items bigger than 1 in 'a'
index = np.where(a > 1)
I was expecting my index to return an empty list. In fact it returns a tuple object, like:
index
Out[5]: (array([], dtype=int64), array([], dtype=int64))
So, the test I was imposing:
#testing if there are values
#in 'a' that fulfil the where condition
if index[0] != []:
print('Values found.')
#testing if there are no values
#in 'a' that fulfil the where condition
if index[0] == []:
print('No values found.')
Will not achieve its purpose because I was comparing different objects (is that correct to say?).
So what is the correct way to create this test?
Thanks for your time!
For your 2D array, np.where returns a tuple of arrays of indices (one for each axis), so that a[index] gives you an array of the elements fulfilling the condition.
Indeed, you compared an empty list to an empty array. Instead, I would compare the size property (or e.g. len()) of the first element of this tuple:
if index[0].size == 0:
print('No values found.')