Calculating across rows and columns at same time - python-3.x

Question

I am trying to do some calculation across rows and columns in python. It is taking painfully longer time to execute for large dataset.
I am trying to do some calculation as follows:
Df =pd.DataFrame({'A': [1,1,1,2,2,2,2],
'unit': [1,2,1,1,1,1,2],
'D1':[100,100,100,200,300,400,3509],
'D2':[200,200,200,300,300,400,2500],
'D3':[50,50,50,60,50,67,98],
'Level1':[1,4,0,4,4,4,5],
'Level2':[45,3,0,6,7,8,9],
'Level3':[0,0,34,8,7,0,5]
})
For each value of A (in above example A=1 and 2) I am running a function sequentially (i.e., I can not run the same function for A=1 and A=2 at the same time since outcome of A=1 changes some other values for A=2). I am calculating a Score as:
def score(data):
data['score_Level1']=np.where(data['Level1']>=data['unit'], data['unit'], 0)*(((np.where(data['Level1']>=data['unit'], data['unit'], 0)).sum()*100) +(10/data['D1']))
data['score_Level2']=np.where(data['Level2']>=data['unit'], data['unit'], 0)*(((np.where(data['Level2']>=data['unit'], data['unit'], 0)).sum()*100) +(10/data['D2']))
data['score_Level3']=np.where(data['Level3']>=data['unit'], data['unit'], 0)*(((np.where(data['Level3']>=data['unit'], data['unit'], 0)).sum()*100) +(10/data['D3']))
return(data)
What above code does is it goes row by row and gives score for Leveli (i=1,2,3) as follows:
Step1:
compare Value of "Leveli' with corresponding "unit" column, if Leveli >=unit then unit else 0.
Step2:
Then it (sums up result for above operation across all rows for Leveli)*100+ (1/Di) = Lets say "S"
Step3:
It goes row by row again and assign a score for Leveli as:
Step1*Step2 (for each row)
Above code should yield results for A=1 as:
score(Df[Df['A']==1])
I am listing only scoring for Level1, same thing happends for Level2 and Level3
Step1:
Compare 1>=1 = True Yields 1, 4>=2 = true Yields 2, 0>=1 =False Yields 0
Step2:
(1+2+0)*100+1/100=300.1
Step3:
Compare 1>=1 = True Yields 1 *300.1=300.1
Compare 4>=2 = True Yields 2 *300.1=600.2
Compare 0>=1 = False Yields 0 *300.1=0
I am doing this activity for 200 million values of A. Since it has to be done sequentially (A=n depends on outcome of A=n-1), it is taking a long time to compute.
Any suggestion of making it faster is much appreciated.

Answer 1

I think, you can avoid the where, which should run faster.
Can you please try this code:
def score2(data, score_field, level_field, d_field):
indexer= data[level_field]>=data['unit']
data[score_field]= 0.0
data.loc[indexer, score_field]= data['unit'] * data.loc[indexer, 'unit'].sum()*100 + 10/data[d_field]
return(data)
score2(Df, 'score_Level1', 'Level1', 'D1')
score2(Df, 'score_Level2', 'Level2', 'D2')
score2(Df, 'score_Level3', 'Level3', 'D3')
The .loc in combination with the indexer replaces the where. On the left side of the assignment it will only set the values for the rows in which the "level-field" is greather than unit. All others stay as they are. Without the line data[score_field]= 0.0 they would contain NaN.
Btw. pandas has it's own .where method, which works on series. It is slightly different from the numpy implementation.