I have the loan dataset below -
Sector
Total Units
Bad units
Bad Rate
Retail Trade
16
5
31%
Construction
500
1100
20%
Healthcare
165
55
33%
Mining
3
2
67%
Utilities
56
19
34%
Other
300
44
15%
How can I create a ranking function to sort this data based on the bad_rate while also accounting for the number of units ?
e.g This is the result when I sort in descending order based on bad_rate
Sector
Total Units
Bad units
Bad Rate
Mining
3
2
67%
Utilities
56
19
34%
Healthcare
165
55
33%
Retail Trade
16
5
31%
Construction
500
1100
20%
Other
300
44
15%
Here, Mining shows up first but I don't really care about this sector as it only has a total of 3 units. I would like construction, other and healthcare to show up on the top as they have more # of total as well as bad units
STEP 1) is easy...
Use SORT("Range","ByColNumber","Order")
Just put it in the top left cell of where you want your sorted data.
=SORT(B3:E8,4,-1):
STEP 2)
Here's the tricky part... you need to decide how to weight the outage.
Here, I found multiplying the Rate% by the Total Unit Rank:
I think this approach gives pretty good results... you just need to play with the formula!
Please let me know what formula you eventually use!
You would need to define sorting criteria, since you don't have a priority based on column, but a combination instead. I would suggest defining a function that weights both columns: Total Units and Bad Rate. Using a weight function would be a good idea, but first, we would need to normalize both columns. For example put the data in a range 0-100, so we can weight each column having similar values. Once you have the data normalized then you can use criteria like this:
w_1 * x + w_2 * y
This is the main idea. Now to put this logic in Excel. We create an additional temporary variable with the previous calculation and name it crit. We Define a user LAMBDA function SORT_BY for calculating crit as follows:
LAMBDA(a,b, wu*a + wbr*b)
and we use MAP to calculate it with the normalized data. For convenience we define another user LAMBDA function to normalize the data: NORM as follows:
LAMBDA(x, 100*(x-MIN(x))/(MAX(x) - MIN(x)))
Note: The above formula ensures a 0-100 range, but because we are going to use weights maybe it is better to use a 1-100 range, so the weight takes effect for the minimum value too. In such case it can be defined as follow:
LAMBDA(x, ( 100*(x-MIN(x)) + (MAX(x)-x) )/(MAX(x)-MIN(x)))
Here is the formula normalizing for 0-100 range:
=LET(wu, 0.6, wbr, 0.8, u, B2:B7, br, D2:D7, SORT_BY, LAMBDA(a,b, wu*a + wbr*b),
NORM, LAMBDA(x, 100*(x-MIN(x))/(MAX(x) - MIN(x))),
crit, MAP(NORM(u), NORM(br), LAMBDA(a,b, SORT_BY(a,b))),
DROP(SORT(HSTACK(A2:D7, crit),5,-1),,-1))
You can customize how to weight each column (via wu for Total Units and wbr for Bad Rates columns). Finally, we present the result removing the sorting criteria (crit) via the DROP function. If you want to show it, then remove this step.
If you put the formula in F2 this would be the output:
How to sort rows when a column has standard electronics "suffixes"?
I see many questions here that are close, but most go the other way, like
Format numbers in thousands (K) in Excel
Anyone in electronics will immediately appreciate this problem. I have lots of parts lists, and am pasting values into Excel/GSheets. They are standard suffixes, but clearly not solely numbers. Here is a representative sample:
A B C D
RA367 0603 2.2 5% 1/10w MF-LF
RA770 0201 5.1k 1% 1/20w MF
RA775 0201 5.1k 1% 1/20w MF
RB600 0402 0 5% 1/16w MF-LF
RB604 0201 0 5% 1/20w MF
Only column C is needed to sort. The suffixes vary on the type of component, but are not mixed when sorted. In other words, you would never sort a column of 'mixed' components such as:
2.5k
1.0pF
10m
20uF
2 kOhms
[...]
The mutiplier portion of the suffixes would always be the same, as in R, k, m, , are typically resistors; pF, F, and uF are capacitors, H, uH, etc. is for inductors (for Henries), etc. So it is best if "conversion" for sorting consider only the first character (u, p, k, m, R) which are always the multiplier, and if no multiplier character (as in the 0 in the first example) just sort as a number.
1.1 = 1.1
1.1 k = 1100
1.1k = 1100
1.1kOhms = 1100
1.1k Ohms = 1100
[...]
This is because lots of parts listings will omit the type of value (resistor, capacitor, etc.) and only give the base number (1, 2, 40, 1m, 2.2k, ...). his is because again, values of different components are never mixed.
Here is a real-world snippet from a large distributor, from a downloaded CSV:
[...]
0 Ohms
100 kOhms
100 kOhms
100 kOhms
1 MOhms
1 MOhms
1 MOhms
100 Ohms
100 Ohms
100 Ohms
49.9 Ohms
[...]
Here you can see how the default sorting on first, second character fails, and that there is even a space between the base and multiplier. A solution should not have to worry about a finite list of types of components, ignoring the Ohms, R, H, F, etc. after the value is determined by the base and optional multiplier.
These are the only two ways you will see components listed-with or without that space. I am wondering if there is a single, elegant function to apply to a range, or if multiple ones are needed based on the space introduced in the second example.
This may seem like an obscure problem, but large suppliers offer CSV downloads of their products, and when you need to order, and are combining lists in different formats, it becomes most cumbersome.
Something like this should work for resistors and capacitors, assuming m meaning milli- isn't used:
=sort(A:A,REGEXEXTRACT(A:A,"[0-9.]+")*1000^(search(iferror(regexextract(A:A,"[0-9.]+\s*([pukmKM])")," "),"pux km")-4),1)
(I know you wouldn't mix them, but this is just to demonstrate)
Like the title, I used online an online data set for stem-leaf plot. But I don't know how to read it. For example, in the line of Stem 7. and Leaf .5555, why Frequency = 18? And what does the line Each leaf: 4 case(s) mean?
Every answer is very helpful to me.
Here is an example.
DATA LIST FREE /x1.
BEGIN DATA.
10 22 22 13 14 10 16 17 17 17
END DATA.
EXAMINE VARIABLES=x1 /PLOT STEMLEAF.
x1 Stem-and-Leaf Plot
Frequency Stem & Leaf
4.00 1 . 0034
4.00 1 . 6777
2.00 2 . 22
Stem width: 10.00
Each leaf: 1 case(s)
In these data, the "Stem" is the tens place of each value and the "Leaf" is the ones place. There are four cases in the first line, representing the values 10, 10, 13, and 14 in the data. That's why "Frequency" is 4; there are four cases. There are only 2 in the last one, for both values of 22 in the original data. As the data get larger, StemLeaf plots can get a little harder to read, but the other real value of them is their shape, which gives you an idea of the shape of the distribution. For another view of that shape, ask SPSS to produce a histogram.
I am trying to calculate percentage growth in excel with a positive and negative number.
This Year's value: 2434
Last Year's value: -2
formula I'm using is:
(This_Year - Last_Year) / Last_Year
=(2434 - -2) / -2
The problem is I get a negative result. Can an approximate growth number be calculated and if so how?
You could try shifting the number space upward so they both become positive.
To calculate a gain between any two positive or negative numbers, you're going to have to keep one foot in the magnitude-growth world and the other foot in the volume-growth world. You can lean to one side or the other depending on how you want the result gains to appear, and there are consequences to each choice.
Strategy
Create a shift equation that generates a positive number relative to the old and new numbers.
Add the custom shift to the old and new numbers to get new_shifted and old_shifted.
Take the (new_shifted - old_shifted) / old_shifted) calculation to get the gain.
For example:
old -> new
-50 -> 30 //Calculate a shift like (2*(50 + 30)) = 160
shifted_old -> shifted_new
110 -> 190
= (new-old)/old
= (190-110)/110 = 72.73%
How to choose a shift function
If your shift function shifts the numbers too far upward, like for example adding 10000 to each number, you always get a tiny growth/decline. But if the shift is just big enough to get both numbers into positive territory, you'll get wild swings in the growth/decline on edge cases. You'll need to dial in the shift function so it makes sense for your particular application. There is no totally correct solution to this problem, you must take the bitter with the sweet.
Add this to your excel to see how the numbers and gains move about:
shift function
old new abs_old abs_new 2*abs(old)+abs(new) shiftedold shiftednew gain
-50 30 50 30 160 110 190 72.73%
-50 40 50 40 180 130 220 69.23%
10 20 10 20 60 70 80 14.29%
10 30 10 30 80 90 110 22.22%
1 10 1 10 22 23 32 39.13%
1 20 1 20 42 43 62 44.19%
-10 10 10 10 40 30 50 66.67%
-10 20 10 20 60 50 80 60.00%
1 100 1 100 202 203 302 48.77%
1 1000 1 1000 2002 2003 3002 49.88%
The gain percentage is affected by the magnitude of the numbers. The numbers above are a bad example and result from a primitive shift function.
You have to ask yourself which critter has the most productive gain:
Evaluate the growth of critters A, B, C, and D:
A used to consume 0.01 units of energy and now consumes 10 units.
B used to consume 500 units and now consumes 700 units.
C used to consume -50 units (Producing units!) and now consumes 30 units.
D used to consume -0.01 units (Producing) and now consumes -30 units (producing).
In some ways arguments can be made that each critter is the biggest grower in their own way. Some people say B is best grower, others will say D is a bigger gain. You have to decide for yourself which is better.
The question becomes, can we map this intuitive feel of what we label as growth into a continuous function that tells us what humans tend to regard as "awesome growth" vs "mediocre growth".
Growth a mysterious thing
You then have to take into account that Critter B may have had a far more difficult time than critter D. Critter D may have far more prospects for it in the future than the others. It had an advantage! How do you measure the opportunity, difficulty, velocity and acceleration of growth? To be able to predict the future, you need to have an intuitive feel for what constitutes a "major home run" and a "lame advance in productivity".
The first and second derivatives of a function will give you the "velocity of growth" and "acceleration of growth". Learn about those in calculus, they are super important.
Which is growing more? A critter that is accelerating its growth minute by minute, or a critter that is decelerating its growth? What about high and low velocity and high/low rate of change? What about the notion of exhausting opportunities for growth. Cost benefit analysis and ability/inability to capitalize on opportunity. What about adversarial systems (where your success comes from another person's failure) and zero sum games?
There is exponential growth, liner growth. And unsustainable growth. Cost benefit analysis and fitting a curve to the data. The world is far queerer than we can suppose. Plotting a perfect line to the data does not tell you which data point comes next because of the black swan effect. I suggest all humans listen to this lecture on growth, the University of Colorado At Boulder gave a fantastic talk on growth, what it is, what it isn't, and how humans completely misunderstand it. http://www.youtube.com/watch?v=u5iFESMAU58
Fit a line to the temperature of heated water, once you think you've fit a curve, a black swan happens, and the water boils. This effect happens all throughout our universe, and your primitive function (new-old)/old is not going to help you.
Here is Java code that accomplishes most of the above notions in a neat package that suits my needs:
Critter growth - (a critter can be "radio waves", "beetles", "oil temprature", "stock options", anything).
public double evaluate_critter_growth_return_a_gain_percentage(
double old_value, double new_value) throws Exception{
double abs_old = Math.abs(old_value);
double abs_new = Math.abs(new_value);
//This is your shift function, fool around with it and see how
//It changes. Have a full battery of unit tests though before you fiddle.
double biggest_absolute_value = (Math.max(abs_old, abs_new)+1)*2;
if (new_value <= 0 || old_value <= 0){
new_value = new_value + (biggest_absolute_value+1);
old_value = old_value + (biggest_absolute_value+1);
}
if (old_value == 0 || new_value == 0){
old_value+=1;
new_value+=1;
}
if (old_value <= 0)
throw new Exception("This should never happen.");
if (new_value <= 0)
throw new Exception("This should never happen.");
return (new_value - old_value) / old_value;
}
Result
It behaves kind-of sort-of like humans have an instinctual feel for critter growth. When our bank account goes from -9000 to -3000, we say that is better growth than when the account goes from 1000 to 2000.
1->2 (1.0) should be bigger than 1->1 (0.0)
1->2 (1.0) should be smaller than 1->4 (3.0)
0->1 (0.2) should be smaller than 1->3 (2.0)
-5-> -3 (0.25) should be smaller than -5->-1 (0.5)
-5->1 (0.75) should be smaller than -5->5 (1.25)
100->200 (1.0) should be the same as 10->20 (1.0)
-10->1 (0.84) should be smaller than -20->1 (0.91)
-10->10 (1.53) should be smaller than -20->20 (1.73)
-200->200 should not be in outer space (say more than 500%):(1.97)
handle edge case 1-> -4: (-0.41)
1-> -4: (-0.42) should be bigger than 1-> -9:(-0.45)
Simplest solution is the following:
=(NEW/OLD-1)*SIGN(OLD)
The SIGN() function will result in -1 if the value is negative and 1 if the value is positive. So multiplying by that will conditionally invert the result if the previous value is negative.
Percentage growth is not a meaningful measure when the base is less than 0 and the current figure is greater than 0:
Yr 1 Yr 2 % Change (abs val base)
-1 10 %1100
-10 10 %200
The above calc reveals the weakness in this measure- if the base year is negative and current is positive, result is N/A
It is true that this calculation does not make sense in a strict mathematical perspective, however if we are checking financial data it is still a useful metric. The formula could be the following:
if(lastyear>0,(thisyear/lastyear-1),((thisyear+abs(lastyear)/abs(lastyear))
let's verify the formula empirically with simple numbers:
thisyear=50 lastyear=25 growth=100% makes sense
thisyear=25 lastyear=50 growth=-50% makes sense
thisyear=-25 lastyear=25 growth=-200% makes sense
thisyear=50 lastyear=-25 growth=300% makes sense
thisyear=-50 lastyear=-25 growth=-100% makes sense
thisyear=-25 lastyear=-50 growth=50% makes sense
again, it might not be mathematically correct, but if you need meaningful numbers (maybe to plug them in graphs or other formulas) it's a good alternative to N/A, especially when using N/A could screw all subsequent calculations.
You should be getting a negative result - you are dividing by a negative number. If last year was negative, then you had negative growth. You can avoid this anomaly by dividing by Abs(Last Year)
Let me draw the scenario.
From: -303 To 183, what is the percentage change?
-303, -100% 0 183, 60.396% 303, 100%
|_________________ ||||||||||||||||||||||||________|
(183 - -303) / |-303| * 100 = 160.396%
Total Percent Change is approximately 160%
Note: No matter how negative the value is, it is treated as -100%.
The best way to solve this issue is using the formula to calculate a slope:
(y1-y2/x1-x2)
*define x1 as the first moment, so value will be "C4=1"
define x2 as the first moment, so value will be "C5=2"
In order to get the correct percentage growth we can follow this order:
=(((B4-B5)/(C4-C5))/ABS(B4))*100
Perfectly Works!
Simplest method is the one I would use.
=(ThisYear - LastYear)/(ABS(LastYear))
However it only works in certain situations. With certain values the results will be inverted.
It really does not make sense to shift both into the positive, if you want a growth value that is comparable with the normal growth as result of both positive numbers. If I want to see the growth of 2 positive numbers, I don't want the shifting.
It makes however sense to invert the growth for 2 negative numbers. -1 to -2 is mathematically a growth of 100%, but that feels as something positive, and in fact, the result is a decline.
So, I have following function, allowing to invert the growth for 2 negative numbers:
setGrowth(Quantity q1, Quantity q2, boolean fromPositiveBase) {
if (q1.getValue().equals(q2.getValue()))
setValue(0.0F);
else if (q1.getValue() <= 0 ^ q2.getValue() <= 0) // growth makes no sense
setNaN();
else if (q1.getValue() < 0 && q2.getValue() < 0) // both negative, option to invert
setValue((q2.getValue() - q1.getValue()) / ((fromPositiveBase? -1: 1) * q1.getValue()));
else // both positive
setValue((q2.getValue() - q1.getValue()) / q1.getValue());
}
These questions are answering the question of "how should I?" without considering the question "should I?" A change in the value of a variable that takes positive and negative values is fairly meaning less, statistically speaking. The suggestion to "shift" might work well for some variables (e.g. temperature which can be shifted to a kelvin scale or something to take care of the problem) but very poorly for others, where negativity has a precise implication for direction. For example net income or losses. Operating at a loss (negative income) has a precise meaning in this context, and moving from -50 to 30 is not in any way the same for this context as moving from 110 to 190, as a previous post suggests. These percentage changes should most likely be reported as "NA".
Just change the divider to an absolute number.i.e.
A B C D
1 25,000 50,000 75,000 200%
2 (25,000) 50,000 25,000 200%
The formula in D2 is: =(C2-A2)/ABS(A2) compare with the all positive row the result is the same (when the absolute base number is the same). Without the ABS in the formula the result will be -200%.
Franco
Use this code:
=IFERROR((This Year/Last Year)-1,IF(AND(D2=0,E2=0),0,1))
The first part of this code iferror gets rid of the N/A issues when there is a negative or a 0 value. It does this by looking at the values in e2 and d2 and makes sure they are not both 0. If they are both 0 then it will place a 0%. If only one of the cells are a 0 then it will place 100% or -100% depending on where the 0 value falls. The second part of this code (e2/d2)-1 is the same code as (this year - lastyear)/Last year
Please click here for example picture
I was fumbling for answers today, and think this would work...
=IF(C5=0, B5/1, IF(C5<0, (B5+ABS(C5)/1), IF(C5>0, (B5/C5)-1)))
C5 = Last Year, B5 = This Year
We have 3 IF statements in the cell.
IF Last Year is 0, then This Year divided by 1
IF Last Year is less than 0, then This Year + ABSolute value of Last Year divided by 1
IF Last Year is greater than 0, then This Year divided by Last Year minus 1
Use this formula:
=100% + (Year 2/Year 1)
The logic is that you recover 100% of the negative in year 1 (hence the initial 100%) plus any excess will be a ratio against year 1.
Short one:
=IF(D2>C2, ABS((D2-C2)/C2), -1*ABS((D2-C2)/C2))
or confusing one (my first attempt):
=IF(D2>C2, IF(C2>0, (D2-C2)/C2, (D2-C2)/ABS(C2)), IF(OR(D2>0,C2>0), (D2-C2)/C2, IF(AND(D2<0, C2<0), (D2-C2)/ABS(C2), 0)))
D2 is this year, C2 is last year.
Formula should be this one:
=(thisYear+IF(LastYear<0,ABS(LastYear),0))/ABS(LastYear)-100%
The IF value if < 0 is added to your Thisyear value to generate the real difference.
If > 0, the LastYear value is 0
Seems to work in different scenarios checked
This article offers a detailed explanation for why the (b - a)/ABS(a) formula makes sense. It is counter-intuitive at first, but once you play with the underlying arithmetic, it starts to make sense. As you get used to it eventually, it changes the way you look at percentages.
Aim is to get increase rate.
Idea is following:
At first calculate value of absolute increase.
Then value of absolute increase add to both, this and last year values. And then calculate increase rate, based on the new values.
For example:
LastYear | ThisYear | AbsoluteIncrease | LastYear01 | ThisYear01 | Rate
-10 | 20 | 30 = (10+20) | 20=(-10+30)| 50=(20+30) | 2.5=50/20
-20 | 20 | 40 = (20+20) | 20=(-20+40)| 60=(20+40) | 3=60/2
=(This Year - Last Year) / (ABS(Last Year))
This only works reliably if this year and last year are always positive numbers.
For example last_year=-50 this_year = -1. You get -100% growth when in fact the numbers have improved a great deal.