I am working with a dataset that is structured like the one below. As you can see, the indicator column contains binary categorical data.
country_code indicator cumulative_count
AFG cases 52909
AFG deaths 2230
... ... ...
I would like to turn the indicator column into two separate columns (corresponding with the values of indicator: cases and deaths). I.e. I'm expecting the final result to be like this:
country_code cases deaths
AFG 52909 2230
... ... ...
Notes:
The original dataset is publically accessible from ECDC website.
I am only interested in the cumulative_count of one specific year_week (2020-53).
Here is a screenshot of the dataset:
This can also be accomplished using Power Query, available in Windows Excel 2010+ and Excel 365 (Windows or Mac)
To use Power Query
Load your data table into Excel
Select some cell in your Data Table
Data => Get&Transform => from Table/Range or from within sheet
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
let
//Read in the table
//Change table name in next line to your actual table name
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
//Remove the unneeded columns
#"Removed Other Columns" = Table.SelectColumns(Source,{"country_code", "indicator", "year_week", "cumulative_count"}),
//Set the data types for those columns
#"Set Data Type" = Table.TransformColumnTypes(#"Removed Other Columns",{
{"country_code", type text}, {"indicator", type text},{"year_week", type text},{"cumulative_count", Int64.Type}
}),
//Pivot the Indicator column and aggregate by Sum
#"Pivoted Column" = Table.Pivot(#"Set Data Type",
List.Distinct(#"Removed Other Columns"[indicator]), "indicator", "cumulative_count", List.Sum),
//Filter to show only the relevant year-week for rows where thiere is a country_code
// (the others refer to continents)
#"Filtered Rows" = Table.SelectRows(#"Pivoted Column", each ([country_code] <> null) and ([year_week] = "2020-53"))
in
#"Filtered Rows"
filtered to show just 2020-53
If I'm understanding your question correctly. one way:
Add new column F
Formula in $F$2: sumifs($D2:$D$9999, $B2:$B$9999, $B2, $E2:$E$9999, "deaths")
copy formula down through end record
filter column E for "cases"
if you then insert rows above the header row, you can use Subtotal(109, ...) to view cumulative counts for a specific year, or alternatively add another column with Sumif as shown above
Related
I have a table of data which is consisted of 18 columns and 2.017 rows. I can get the row that has the highest (MAX) value in a cell but I need the row that has the most cells with higher values and have them in DESC order. I haven't managed yet to find a relevant post to this.
Here follows an example:
Using numbers up to 10 for illustration, the following shows the logic behind. (The actual numbers are those shown in Exhibit1)
Thank you
EDIT:
I am adding the below in order to try to clarify further. I am not sure if it is the correct path to go but I hope it makes sense.
In Exhibit2 I am indexing each column Desc (Based on Exhibit1) and then =SUM in the end of the row. Following this logic, the name having the lowest total is the one with the most high values (not the highest) in its row.
The result table is the following
Although possible with formulas and helper tables/columns, this can also be accomplished using Power Query, available in Windows Excel 2010+ and Excel 365 (Windows or Mac)
To use Power Query
Select some cell in your Data Table
Data => Get&Transform => from Table/Range or from within sheet
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
As we discussed in our Chat, I transform each column into a list of Ranked Entries; then sum the ranks for each row and sort as you have laid out.
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table5"]}[Content],
//type all the columns
data = Table.TransformColumnTypes(Source,{
{"Order", Int64.Type},
{"Name", type text}} &
List.Transform(List.RemoveFirstN(Table.ColumnNames(Source),2), each {_, type number})
),
//Replace with ranks
//generate list of transforms to dynamically include all columns
cols = List.RemoveFirstN(Table.ColumnNames(data),2),
xForms = List.Transform(cols, (c)=> {c, each List.PositionOf(List.Sort(Table.Column(data,c),Order.Descending),_)}),
ranks = Table.TransformColumns(data,xForms),
//add Index column to enable row-wise sums
// then add the sumRank column and delete the Index column
#"Added Index" = Table.AddIndexColumn(ranks, "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "sumRank", each
List.Sum(
Record.ToList(
Record.RemoveFields(#"Added Index"{[Index]},{"Order","Name","Index"})
)
)),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index"}),
//join back with the original data table
//extract the sumRank column
join = Table.NestedJoin(data,{"Order","Name"}, #"Removed Columns",{"Order","Name"}, "joined",JoinKind.FullOuter),
#"Expanded joined" = Table.ExpandTableColumn(join, "joined", {"sumRank"}, {"sumRank"}),
//sort by the sumRank column, then remove it
#"Sorted Rows" = Table.Sort(#"Expanded joined",{{"sumRank", Order.Ascending}}),
#"Removed Columns1" = Table.RemoveColumns(#"Sorted Rows",{"sumRank"})
in
#"Removed Columns1"
This set-up is volatile, so I would only adopt it if non-volatile alternatives are not forthcoming.
An additional column in your table with the following formula:
=SUM(COUNTIF(OFFSET([Column1],,TRANSPOSE(ROW(INDIRECT("1:"&COLUMNS(Table1[#[Column1]:[Column4]])))-1)),">="&Table1[#[Column1]:[Column4]]))
which you can then use to sort your table.
Note that this formula will most likely require committing with CTRL+SHIFT+ENTER for your version of Excel.
Amend the table and column names as required, noting that the part
Table1[#[Column1]:[Column4]]
as well as including the table name, should comprise the leftmost and rightmost of the contiguous columns to be interrogated.
Sorry if this has been asked before, but I was unable to find anything specific to this need, probably due to it being odd to phrase. Essentially I have a table like this that I would like to transform into the table below:
zip
blue
green
10000
1
2
zip
color
10000
blue
10000
green
Ideally, I would like to do this all purely in SQL or purely in Excel, but eventually I will want to transpose through R or Python once I get more familiar.
You can do this easily in Power Query, available in Windows Excel 2010+, and Excel 365.
Select some single cell in your source table
Data => Get Data => from within sheet
In the PQ UI, select the Zip column
Transform => Unpivot => Unpivot other columns
Delete the Values column
Rename the Attribute Column => Color
Home => Close & Load
Here is M-Code that will do the same thing, with some changes so as not to have to hard-code other colors that you might have besides the two you show.
You would paste this into the Advanced Editor of PQ; change the Table name at the top. Then read the comments and explore the Applied Steps to better understand the algorithm.
let
//Change table name in next line to the actual name in your workbook
Source = Excel.CurrentWorkbook(){[Name="Table29"]}[Content],
//set data types
// Zip is text to retain leading zero's
// Others are all integers
#"Changed Type" = Table.TransformColumnTypes(Source,
{{"zip", type text}} & List.Transform(List.RemoveFirstN(Table.ColumnNames(Source),1), each {_, Int64.Type})),
//Unpivot all columns except for the Zip column
// And name the "color" column as "Color"
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"zip"}, "Color", "Value"),
//Remove the value column since you do not show it in your result example
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Other Columns",{"Value"})
in
#"Removed Columns"
My issue is the following: I have a table where I have multiple columns that have date and values but represent different things. Here is an example for my headers:
I Customer name I Type of Service I Payment 1 date I Payment 1 amount I Payment 2 date I Payment 2 amount I Payment 3 date I Payment 3 amount I Payment 4 date I Payment 4 amount I
What I want to do is sumifs the table based on multiple criteria. For example:
I Type of Service I Month 1 I Month 2 I Month 3 I Month 4
Service 1
Service 2
Service 3
The thing is that I do not want to write 4 sumifs (in this case, but in fact I have more that 4 sets of date:value columns).
I was thinking of creating a new table where I could put all the columns below each other (in one table with 4 columns - Customer name, Type of Service, Date and Payment) but the table should be dynamically created, meaning that it should be expanded dynamically with the new entries in the original table (i.e. if the original table has 200 entries, this would make the new table with 4x200=800 entries, if the original table has one more record then the new table should have 4x201=804 records).
I also checked the PowerQuery option but could not get my head around it.
So any help on the matter will be highly appreciated.
Thank you.
You can certainly create your four column table using Power Query. However, I suspect you may be able to also generate your final report using PQ, so you could add that to this code, if you wish.
And it will update but would require a "Refresh" to do the updating.
The "Refresh" could be triggered by
User selecting the Data/Refresh option
A button on the worksheet which user would have to press.
A VBA event-triggered macro
In any event, in order to make the query adaptable to different numbers of columns requires more M-Code than can be generated from the UI, a well as a custom function.
The algorithm below depends on the data being in this format:
Columns 1 and 2 would be Customer | Type of Service
Remaining columns would alternate between Date | Amount and be Labelled: Payment N Date | Payment N Amount where N is some number
If the real data is not in that format, some changes to the code may be necessary.
To use Power Query:
Select some cell in your Data Table
Data => Get&Transform => from Table/Range
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
To enter the Custom Function, while in the PQ Editord
Right click in the Queries Pane
Add New Query from Blank Query
Paste the custom function code into the Advanced Editor
rename the Query fnPivotAll
M Code
let
//Change Table name in next line to be the Actual table name in your workbook
Source = Excel.CurrentWorkbook(){[Name="Table8"]}[Content],
/*set datatypes dynamically with
first two columns as Text
and subsequent columns alternating as Date and Currency*/
textType = List.Transform(List.FirstN(Table.ColumnNames(Source),2), each {_,Text.Type}),
otherType = List.RemoveFirstN(Table.ColumnNames(Source),2),
dateType = List.Transform(
List.Alternate(otherType,1,1,1), each {_, Date.Type}),
currType = List.Transform(
List.Alternate(otherType,1,1,0), each {_, Currency.Type}),
colTypes = List.Combine({textType, dateType, currType}),
typeIt = Table.TransformColumnTypes(Source,colTypes),
//Unpivot all except first two columns
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(typeIt, List.FirstN(Table.ColumnNames(Source),2), "Attribute", "Value"),
//Remove "Payment n " from attribute column
remPmtN = Table.TransformColumns(#"Unpivoted Other Columns",{{"Attribute", each Text.Split(_," "){2}, Text.Type}}),
//Pivot on the Attribute column without aggregation using Custom Function
pivotAll = fnPivotAll(remPmtN,"Attribute","Value"),
typeIt2 = Table.TransformColumnTypes(pivotAll,{{"date", Date.Type},{"amount", Currency.Type}})
in
typeIt2
Custom Function: fnPivotAll
//credit: Cam Wallace https://www.dingbatdata.com/2018/03/08/non-aggregate-pivot-with-multiple-rows-in-powerquery/
(Source as table,
ColToPivot as text,
ColForValues as text)=>
let
PivotColNames = List.Buffer(List.Distinct(Table.Column(Source,ColToPivot))),
#"Pivoted Column" = Table.Pivot(Source, PivotColNames, ColToPivot, ColForValues, each _),
TableFromRecordOfLists = (rec as record, fieldnames as list) =>
let
PartialRecord = Record.SelectFields(rec,fieldnames),
RecordToList = Record.ToList(PartialRecord),
Table = Table.FromColumns(RecordToList,fieldnames)
in
Table,
#"Added Custom" = Table.AddColumn(#"Pivoted Column", "Values", each TableFromRecordOfLists(_,PivotColNames)),
#"Removed Other Columns" = Table.RemoveColumns(#"Added Custom",PivotColNames),
#"Expanded Values" = Table.ExpandTableColumn(#"Removed Other Columns", "Values", PivotColNames)
in
#"Expanded Values"
Sample Data
Output
If this does not give you what you require, or if you have issues going further with it to generate your desired reports, post back.
I am using excel 2007
I have a excel sheet with around 1200 records with following structure...
WHAT CAN BE EASIEST WAY TO DO THIS ?
For easy understanding, Adding image :
As per your comment request, here is a Power Query solution.
To enter the code:
Select some cell in your Data Table
Data => Get&Transform => from Table/Range
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Algorithm
Fill in (fill down) the blank rows for the District and Branch columns
Group by District and Branch
For each Group, extract as a delimited string the entries for President, Secretary and Treasurer.
Create the appropriate column names and split the delimited strings into separate columns.
If you have more officers, or more items per officer/position, or more columns before you get to the officer columns, it should be relatively simple to modify the code to take that into account.
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table16"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"District", Text.Type}, {"Branch", type text},
{"President", type text}, {"Secretary", type text}, {"Treasurer", type text}}),
#"Filled Down" = Table.FillDown(#"Changed Type",{"District", "Branch"}),
#"Grouped Rows" = Table.Group(#"Filled Down", {"District", "Branch"},{
{"President", each Text.Combine([President],";")},
{"Secretary", each Text.Combine([Secretary],";")},
{"Treasurer", each Text.Combine([Treasurer],";")}
}),
colHeaderSuffix = {"","Addr","Mobile"},
PresidentCols = List.Accumulate(colHeaderSuffix, {}, (state, current) => List.Combine({state, {"President " & current}})),
#"Split Column by Delimiter" = Table.SplitColumn(#"Grouped Rows", "President",
Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), PresidentCols),
SecretaryCols = List.Accumulate(colHeaderSuffix, {}, (state, current) => List.Combine({state, {"Secretary " & current}})),
#"Split Column by Delimiter2" = Table.SplitColumn(#"Split Column by Delimiter", "Secretary",
Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), SecretaryCols),
TreasurerCols = List.Accumulate(colHeaderSuffix, {}, (state, current) => List.Combine({state, {"Treasurer " & current}})),
#"Split Column by Delimiter3" = Table.SplitColumn(#"Split Column by Delimiter2", "Treasurer",
Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), TreasurerCols)
in
#"Split Column by Delimiter3"
Original Data
Results
Formula Used as follows (Giving solution here so in future members can use it)
The formula is : In empty cell after Treasurer Column,
=If($a2="","",a2) // copy over next 4 columns to give the District, Branch, Pre name, address, mobile elements as =If($a2="","",a2), =If($b2="","",b2), =If($c2="","",c2), =If($c2="","",c3), =If($c2="","",c3), =If($c2="","",c4)
=if($d2="","",d2) // copy over next 2 columns for Secretary details like =if($d2="","",d2), =if($d2="","",d3), =if($d2="","",d4)
=if($g2="","",g2) // copy over next 2 columns for Treasurer details like =if($g2="","",g2), =if($g2="","",g3), =if($g2="","",g4)
Now Select All New Formula Cells in a row after Treasurer column >> Drag Down Till All Records....
Then Copy all these down to the bottom of your data
Either copy / paste special >> values to somewhere else and
Then sort by District / Branch / Pres to drop the blank rows
I don't know if your excel pc will be able to handle it but you can use the [Paste Transpose][1].
You copy everything, (my advice, go to a new spreadsheet, but you can use the same one),
and then you past it use past transform
*edit
after you edited your question with the example you might want to use the past transform and then use pivot table
In my data, I have 2 columns who represent a country visited before and a country visited after the cities that I am studying.
Here's a picture of my data sample: https://i.imgur.com/kS4K9uK.png
I'd like to represent in my pivot table all the countries linked to each city (so before and after the city). I'd like to have the cities in my line and all the countries who can possibly be visited before and after as my columns and the count of those in my values.
Here is a picture of what I'd like to achieve, but I can only do it for one of the columns (country after in that case). I'd like the same format but having the data of both before and after (but it's important to know that it's not necessarily the same countries in the 2 columns so I can't just have one of the country columns as the head and both as the values): https://i.imgur.com/PUjhSmB.png
When I place the cities in the line and the 2 country columns in value and columns, it is so difficult to read the table as the before and after are all separate and might even be counted as a pair. and if they are not in the pivot table column they only give me the count of countries before and after but not by the countries, which is not what I'm looking for.
Here is a picture of the result of the pivot table: https://i.imgur.com/3j4BD3k.png
I also tried to create a new field by doing «Country before» + «Country after» but it doesn't seem to work as the data is in text.
Ok I think understand the output now. You essentially want a count of the number of occurrences of each country in columns B+C, grouped by the city. I'll provide a few ways so you can select what suits you best.
Simplest method
The easiest way I can think of is simply paste the second column under the first column and then pivot on this new table.
COUNTIF
A more repeatable way would be to essentially make your own pivot table and use the COUNTIF function to count the instances of each country.
=COUNTIFS($A$1:$A$6,$F2,$B$1:$B$6,G$1)+COUNTIFS($A$1:$A$6,$F2,$C$1:$C$6,G$1)
Power Query
The most repeatable way is to use PowerQuery. This will enable you to refresh the data at the click of a button. To do this (assuming you have excel 2016) go to the Data tab and, with you data selected click "From Table/Range". The Power Query window will open. On the top left of the screen will be a button with advanced editor. When you open it you'll see the following code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"City", type text}, {"Country Before", type text}, {"Country After", type text}})
in
#"Changed Type"
Replace the code with the following code. Note that your table may be called something different. You can see what it's called on the second line of the code. The code below uses "Table1" - you can replace this with the name of your table.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"City", type text}, {"Country Before", type text}, {"Country After", type text}}),
#"Before" = Table.RemoveColumns(#"Changed Type",{"Country Before"}),
#"After" = Table.RemoveColumns(#"Changed Type",{"Country After"}),
#"Append" = Table.Combine({#"Before",#"After"}),
#"Inserted Merged Column" = Table.AddColumn(Append, "Country", each Text.Combine({[Country After], [Country Before]}, ""), type text),
#"Removed Columns" = Table.RemoveColumns(#"Inserted Merged Column",{"Country After", "Country Before"})
in
#"Removed Columns"
Hope that helps.