I'm trying to apply the ranksums function over all rows of two arrays
groupA = subset.iloc[:,4:7].values
groupB = subset.iloc[:,7:10].values
np.apply_along_axis(ranksums, 1, (zip(groupA,groupB)))
I get: AxisError: axis 1 is out of bounds for array of dimension 0
Related
I have a dataframe with a column that contains tuples. I would like to create a new column that extracts the smallest tuple from the tuple column.
What I have tried so far
mydataframe['min_values'] = mydataframe['tuple_column'].apply(lambda x: min(x))
This above approach seems to work when I have at least 2 tuples, but it fails when I only have one tuple e.g. 5 in the example below. Could you guys please suggest a method that would help me accomplish this task in a better manner?
Example and desired result
Tuple Column
New Column
(1,2,3,5)
1
(10,11)
10
(5)
5
Thanks
(5) is not a tuple, this is 5. Use numpy.min that handles scalar values as input:
import numpy as np
df['New Column'] = df['Tuple Column'].apply(np.min)
Output:
Tuple Column New Column
0 (1, 2, 3, 5) 1
1 (10, 11) 10
2 5 5
Here is a way using map()
df['Tuple Column'].map(lambda x: min(x) if isinstance(x,tuple) else x)
Output:
0 1
1 3
2 5
df1.applymap(lambda x:pd.Series(eval(x)).min())
Output:
0 1
1 3
2 5
I am looking for a quick way to generate a long dataframe. For example, the input is:
Column "color": [1,2,3] (length: 3)
Column "weekday": [0,1] (length: 2)
The expected output is:
color weekday
1 0
2 0
3 0
1 1
2 1
3 1
And this output dataframe has the length as 2*3 = 6.
Is there a quick way to generate such dataframes based on the series as the input? And it is possible that there are many columns. Thanks.
I have one dataframe with two columns , A and B . first i need to make empty bins with step 1 from 1 to 11 , (1,2),(2,3)....(10,11). then check from original dataframe if column B value greater than 3 then get value of column 'A' 2 rows before when column B is greater than 3.
Here is example dataframe :
df=pd.DataFrame({'A':[1,8.5,5.2,7,8,9,0,4,5,6],'B':[1,2,2,2,3.1,3.2,3,2,1,2]})
Required output 1:
df_out1=pd.DataFrame({'Value_A':[8.5,5.2]})
Required_output_2:
df_output2:
Bins count
(1 2) 0
(2,3) 0
(3,4) 0
(4,5) 0
(5,6) 1
(6,7) 0
（7,8） 0
（8,9） 1
（9,10） 0
（10,11） 0
You can index on a shifted series to get the two rows before 'A' satisfies some condition like
out1 = df['A'].shift(3)[df['B'] > 3]
The thing you want to do with the bins is known as a histogram. You can easily do this with numpy like
count, bin_edges = np.histogram(out1, bins=[i for i in range(1, 12)])
out2 = pd.DataFrame({'bin_lo': bin_edges[:-1], 'bin_hi': bin_edges[1:], 'count': count})
Here 'bin_lo' and 'bin_hi' are the lower and upper bounds of the bins.
I have a messy dataframe where I am trying to "flag" the rows that contain a certain number in the ids column. The values in this column represent an inclusive range: for example, "row 4" contains the following numbers:
2409,2410,2411,2412,2413,2414,2377,2378,1478,1479,1480,1481,1482,1483,1484 And in "row 0" and "row 1" the range for one of the sets is backwards (1931,1930,1929)
If I want to know which rows have sets that contain "2340" and "1930" for example, how would I do this? I think a loop is needed, sometimes will need to query more than just two numbers. Using Python 3.8.
Example Dataframe
x = ['1331:1332,1552:1551,1931:1928,1965:1973,1831:1811,1927:1920',
'1331:1332,1552:1551,1931:1929,180:178,1966:1973,1831:1811,1927:1920',
'2340:2341,1142:1143,1594:1593,1597:1596,1310,1311',
'2339:2341,1142:1143,1594:1593,1597:1596,1310:1318,1977:1974',
'2409:2414,2377:2378,1478:1484',
'2474:2476',
]
y = [6.48,7.02,7.02,6.55,5.99,6.39,]
df = pd.DataFrame(list(zip(x, y)), columns =['ids', 'val'])
display(df)
Desired Output Dataframe
I would write a function that perform 2 steps:
Given the ids_string that contains the range of ids, list all the ids as ids_num_list
Check if the query_id is in the ids_num_list
def check_num_in_ids_string(ids_string, query_id):
# Convert ids_string to ids_num_list
ids_range_list = ids_string.split(',')
ids_num_list = set()
for ids_range in ids_range_list:
if ':' in ids_range:
lower, upper = sorted(ids_range.split(":"))
num_list = list(range(int(lower), int(upper)+ 1))
ids_num_list.update(num_list)
else:
ids_num_list.add(int(ids_range))
# Check if query number is in the list
if int(query_id) in ids_num_list:
return 1
else:
return 0
# Example usage
query_id_list = ['2340', '1930']
for query_id in query_id_list:
df[f'n{query_id}'] = (
df['ids']
.apply(lambda x : check_num_in_ids_string(x, query_id))
)
which returns you what you require:
ids val n2340 n1930
0 1331:1332,1552:1551,1931:1928,1965:1973,1831:1... 6.48 0 1
1 1331:1332,1552:1551,1931:1929,180:178,1966:197... 7.02 0 1
2 2340:2341,1142:1143,1594:1593,1597:1596,1310,1311 7.02 1 0
3 2339:2341,1142:1143,1594:1593,1597:1596,1310:1... 6.55 1 0
4 2409:2414,2377:2378,1478:1484 5.99 0 0
5 2474:2476 6.39 0 0
I am a beginner in Keras and need help to understand keras.argmax(a, axis=-1) and keras.max(a, axis=-1). What is the meaning of axis=-1 when a.shape = (19, 19, 5, 80)? And also what will be the output of keras.argmax(a, axis=-1) and keras.max(a, axis=-1)?
This means that the index that will be returned by argmax will be taken from the last axis.
Your data has some shape (19,19,5,80). This means:
Axis 0 = 19 elements
Axis 1 = 19 elements
Axis 2 = 5 elements
Axis 3 = 80 elements
Now, negative numbers work exactly like in python lists, in numpy arrays, etc. Negative numbers represent the inverse order:
Axis -1 = 80 elements
Axis -2 = 5 elements
Axis -3 = 19 elements
Axis -4 = 19 elements
When you pass the axis parameter to the argmax function, the indices returned will be based on this axis. Your results will lose this specific axes, but keep the others.
See what shape argmax will return for each index:
K.argmax(a,axis= 0 or -4) returns (19,5,80) with values from 0 to 18
K.argmax(a,axis= 1 or -3) returns (19,5,80) with values from 0 to 18
K.argmax(a,axis= 2 or -2) returns (19,19,80) with values from 0 to 4
K.argmax(a,axis= 3 or -1) returns (19,19,5) with values from 0 to 79