To count the particular value of given column
Use pd.crosstab with df.sum:
In [236]: output = pd.crosstab(df['Rel_ID'], df['Values'])
In [238]: output['total'] = output.sum(axis=1)
In [239]: output
Out[239]:
Values 400.0 500.0 1700.0 6300.0 total
Rel_ID
TESTA 1 1 1 1 4
TESTB 1 0 1 1 3
TESTC 0 1 1 0 2
TESTD 1 0 1 1 3
TESTE 1 1 0 0 2
Related
I have a df as shown below
df:
Id Jan20 Feb20 Mar20 Apr20 May20 Jun20 Jul20 Aug20 Sep20 Oct20 Nov20 Dec20 Amount
1 20 0 0 12 1 3 1 0 0 2 2 0 100
2 0 0 2 1 0 2 0 0 1 0 0 0 500
3 1 2 1 2 3 1 1 2 2 3 1 1 300
From the above I would like to calculate Activeness value which is the number of non zero columns in the month columns as given below.
'Jan20', 'Feb20', 'Mar20', 'Apr20', 'May20', 'Jun20', 'Jul20',
'Aug20', 'Sep20', 'Oct20', 'Nov20', 'Dec20'
Expected Output:
Id Jan20 Feb20 Mar20 Apr20 May20 Jun20 Jul20 Aug20 Sep20 Oct20 Nov20 Dec20 Amount Activeness
1 20 0 0 12 1 3 1 0 0 2 2 0 100 7
2 0 0 2 1 0 2 0 0 1 0 0 0 500 4
3 1 2 1 2 3 1 1 2 2 3 1 1 300 12
I tried below code:
df['Activeness'] = pd.Series(index=df.index, data=np.count_nonzero(df[['Jan20', 'Feb20',
'Mar20', 'Apr20', 'May20', 'Jun20', 'Jul20',
'Aug20', 'Sep20', 'Oct20', 'Nov20', 'Dec20']], axis=1))
which is working well, but I would like to know is there any method that is faster than this.
You can try:
df['Activeness'] = df.filter(like = '20').ne(0, axis =1).sum(1)
I'm sure there's a simple solution to this but I'm struggling. I want to set the values of a csv file I've created to 1s and 0s so that I can work out the probability based on each row.
Here's the csv data:
0 1 2 3 4 5 6 7 \
0 Reference China Greece Japan S Africa S Korea Sri lanka Taiwan
1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 1
3 1 1 1 1 1 1 1 1
4 1 1 1 1 1 1 1 1
... ... ... ... ... ... ... ... ...
14898 1 1 1 1 1 1 1 1
14899 1 1 1 1 1 1 1 1
14900 1 1 1 1 1 1 1 1
14901 1 1 1 1 1 1 1 1
14902 1 1 1 1 1 1 1 1
8 9 10 11 12 13 14 15 16
0 USA Ecuador Egypt Ghana India Isreal Pakistan Taiwan USA Ohio
1 1.031 1 1 1 1 1 1 1 1
2 1.031 1 1 1 1 1 1 1 1
3 1.031 1 1 1 1 1 1 1 1
4 1.031 1 1 1 1 1 1 1 1
... ... ... ... ... ... ... ... ... ...
14898 1 1 1 1 1 1 1 1 1
14899 1 1 1 1 1 1 1 1 1
14900 1 1 1 1 1 1 1 1 1
14901 1 1 1 1 1 1 1 1 1
14902 1 1 1 1 1 1 1 1 1
[14903 rows x 17 columns]
And I've tried this:
data = pd.DataFrame(pd.read_csv('IEratios.csv', header=None, sep=','))
for x in data:
if x == 1:
x = 0
else:
x = 1
Which I thought would be simple and work but I was wrong and everywhere I look nothing I find seems to apply to all columns and rows, so I am lost.
You can use the .map() function in pandas, this allows you to run a function trough an entire DF column like so:
def changeNumber(x):
if x == 1:
return 0
else:
return 1
df = pd.read_csv('IEratios.csv', sep=',')
df['China'] = df['china'].map(changeNumber)
I don't know if I understand what you want to do.
Do you want to replace the values that are one by zero, and the zeros by one?
If I understand correctly, how are you using panda you can use the following statement
df.replace({"0": "1", "1": "0"}, inplace=True)
You have to be careful with the data type of your dataframe
Have you tried using the numpy.where function?
data = pd.DataFrame(pd.read_csv('IEratios.csv', header=None, sep=','))
data = np.where((data == 1), 0, 1)
df:
Col_A
0 011011
1 011011
2 011011
3 011011
4 011011
How to create a column based on string position, for example in column_A I need to check 0 position create a column B.
Output;
Col_A Col_B
0 011011 pos1,pos4
1 000111 pos1,pos2,pos3
2 011000 pos1,pos4,pos5,pos6
3 011111 pos1
4 011010 pos1,pos4,pos6
First convert strings to DataFrame and add columns names by function in rename:
f = lambda x: f'pos{x+1}'
df1 = pd.DataFrame([list(x) for x in df['Col_A']], index=df.index).rename(columns=f)
print (df1)
pos1 pos2 pos3 pos4 pos5 pos6
0 0 1 1 0 1 1
1 0 0 0 1 1 1
2 0 1 1 0 0 0
3 0 1 1 1 1 1
4 0 1 1 0 1 0
Then compare '0' values by DataFrame.eq and for new column use matrix multiplication by DataFrame.dot with remove separator by Series.str.rstrip:
df['Col_B'] = df1.eq('0').dot(df1.columns + ',').str.rstrip(',')
print (df)
Col_A Col_B
0 011011 pos1,pos4
1 000111 pos1,pos2,pos3
2 011000 pos1,pos4,pos5,pos6
3 011111 pos1
4 011010 pos1,pos4,pos6
I have a dataframe in pandas, an example of which is provided below:
Person appear_1 appear_2 appear_3 appear_4 appear_5 appear_6
A 1 0 0 1 0 0
B 1 1 0 0 1 0
C 1 0 1 1 0 0
D 0 0 1 0 0 1
E 1 1 1 1 1 1
As you can see 1 and 0 occurs randomly in different columns. It would be helpful, if anyone can suggest me a code in python such that I am able to find the column number where the 1 0 0 pattern occurs for the first time. For example, for member A, the first 1 0 0 pattern occurs at appear_1. so the first occurrence will be 1. Similarly for the member B, the first 1 0 0 pattern occurs at appear_2, so the first occurrence will be at column 2. The resulting table should have a new column named 'first_occurrence'. If there is no such 1 0 0 pattern occurs (like in row E) then the value in first occurrence column will the sum of number of 1 in that row. The resulting table should look something like this:
Person appear_1 appear_2 appear_3 appear_4 appear_5 appear_6 first_occurrence
A 1 0 0 1 0 0 1
B 1 1 0 0 1 0 2
C 1 0 1 1 0 0 4
D 0 0 1 0 0 1 3
E 1 1 1 1 1 1 6
Thank you in advance.
I try not to reinvent the wheel, so I develop on my answer to previous question. From that answer, you need to use additional idxmax, np.where, and get_indexer
cols = ['appear_1', 'appear_2', 'appear_3', 'appear_4', 'appear_5', 'appear_6']
df1 = df[cols]
m = df1[df1.eq(1)].ffill(1).notna()
df2 = df1[m].bfill(1).eq(0)
m2 = df2 & df2.shift(-1, axis=1, fill_value=True)
df['first_occurrence'] = np.where(m2.any(1), df1.columns.get_indexer(m2.idxmax(1)),
df1.shape[1])
Out[540]:
Person appear_1 appear_2 appear_3 appear_4 appear_5 appear_6 first_occurrence
0 A 1 0 0 1 0 0 1
1 B 1 1 0 0 1 0 2
2 C 1 0 1 1 0 0 4
3 D 0 0 1 0 0 1 3
4 E 1 1 1 1 1 1 6
If I have a database
Example:
Name A B C
0 Jon 0 1 0
1 Jon 1 0 1
2 Alan 1 0 0
3 Shaya 0 1 1
If there is a duplicate in my dataset I want the person who has column A as 1 to have precedence.
NB. Column A can only have values 1 or 0
Output:
Name A B C
1 Jon 1 0 1
2 Alan 1 0 0
3 Shaya 0 1 1
IIUC sort value before drop duplicate
df.sort_values('A').drop_duplicates('Name',keep='last').sort_index()
Out[126]:
Name A B C
1 Jon 1 0 1
2 Alan 1 0 0
3 Shaya 0 1 1