Extracting the upper quartile data from an array and placing it in new column - excel

Question

Hi and thanks for any help with this in advance
Below is a hypothetical data set; abundance = count data; mud% = the mud content in which the animals were found; mud bin = bins i've made up depending on the mud%; and UQ = upper quartile of the abundance data from its corresponding mud bin (i.e. the upper quartile for the abundance data in mud bin 1 is 17.25 etc).
Problem:
In excel, for abundance data in each of the four mud bins, I'm wanting to extract any values in the abundance column that are >= the upper quartile value for that particular mud bin and place these in a new column on the same sheet (with no gaps between rows from values that didn't meet the criteria) along with their corresponding mud% value in the neighboring cell. I've added the new columns to the below sheet to give you an idea of what I'm after.
abundance | mud% | mud bin | UQ | | New column | Mud% |
18 10.9 1 18 10.9(mud bid 1)
15 6.5 1 44 38.9(mud bin 1)
6 13.4 1 45 38 (mud bin 2)
13 42.1 1 37 37.8(mud bin 2)
15 36.4 1 etc
44 38.9 1 17.25 etc
22 46 2
30 36.4 2
45 38 2
29 35.3 2
37 37.8 2
29 41.8 2 35.25
11 44.4 3
17 47.8 3
21 40.7 3
15 13.9 3
35 13.9 3
14 13.9 3
15 13.9 3 19
19 12 4
14 12 4
10 12 4
12 12 4
14 12 4
13 12 4
45 9.525 4
66 9.525 4
78 9.525 4 45
The reality is I have a rather large dataset containing abundance data for a number of species, all on the same excel sheet and would greatly appreciate any insight into how I might achieve this in the most efficient manor.

Answer 1

For starters, to make this explanation simpler, I will assume that the last row of data is in row 100.
Populate Upper Quartile values for all line items
First you'll need to use the Quartile formula; however, since you want to find the upper quartile within a bin, you'll have to use an array formula. Put this formula in your UQ column (place in cell D2 and drag down). When entering the formula Be sure to press Ctrl+Shift before pressing Enter
=QUARTILE(IF($C$2:$C$100=C2,$A$2:$A$100,""),3)
The first part of this formula, $C$2:$C$100=C2 is your condition. Everywhere this condition is met, you will get the corresponding value in $A$2:$A$100; otherwise, you'll get a blank value. This will give you an array of abundance values that matches the indicated mudbin, C2. now that you have your subset of data, the quartile function will give you the value in the 3rd quartile (17.25 for mudbin 1, which will be placed next to every row that has a mudbin of 1).
Now that we have all the quartiles, we can get all the abundance values that are greater than the UQ for that mudbin. This is done in two parts
Get abundance values greater than mudbin UQ
First, you need to select one column of cells that has the same number of rows as your data (for example, select cells F2:F100)
Enter the following formula into the formula bar (while F2:F100 are highlighted) and press Ctrl+Shift, then enter
=IF($A$2:$A$100>$D$2:$D$100,$A$2:$A$100,"")
Similar to the IF statement used before, this formula finds all the abundance values that are greater than their corresponding UQ value. Now column F will have an abundance number where it is greater than it's UQ value, and a blank where it is not. Now onto the final step.
Populate abundance values that are greater than the UQ value, without the blanks
Select G2:G100 (your "New Column" in your sample data)
Enter the following formula into the formula bar (while G2:G100 are highlighted) and press Ctrl+Shift, then enter
=INDEX(F2:F100,SMALL(IF(F2:F100<>"",ROW(F2:F100)-1),ROW()-ROW($F$1)))
Looking at the IF statement again, this will find every value in F2:F100 that is not blank, but instead of grabbing the values, we'll keep track of the row number of that non blank value (done by ROW(F2:F100)-1
). Now that we have the row numbers of all the non blank values, we can grab the non-blank values in order and populate them in G2:G100. ROW()-ROW($F$1) is a counter, and SMALL will use the counter to determine the nth smallest number to return. Once we have our row number of the non blank value, INDEX returns that value
Finally, to populate the Mud%, you'll need to use the row number of the non blank values to get the mud% and the mud bin (You have the formula already to get the row number of the non blank value).
It's not a simple answer, but at least you won't have to use VBA.