I am a bit new to powerquery and need some help with the points specified below the code.
I have a piece of working code that does the following:
Load the source
find the max "series" of a row, the format is a mix of letters and numbers, i.e. Y21Q3S1, the letters stay the same and the numbers are increasing (year, quarter, and series).
I want to look if a certain tag is assigned to a row, so I search all the tag columns if a tag is present and write that in the "Tags" column and "none" if there were none found
through grouping I find the points per tag, for each "max series"
I finally present it in a table in excel with first column being the series, then a column for the Tags as well as a column "None" if none of the Tags were present. I add a last updated date column.
The code:
let
Source = Csv.Document(Web.Contents("somefile.csv"),[Delimiter=",", Columns=32, Encoding=65001, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type with Locale" = Table.TransformColumnTypes(#"Promoted Headers", {{"Custom field (Points)", type number}}, "en-GB"),
#"Added Custom" = Table.AddColumn(#"Changed Type with Locale", "Max Series", each List.Max({[Series], [Series_1], [Series_2], [Series_3], [Series_4]})),
Tags = Table.AddColumn(#"Added Custom", "Tags", each if List.Contains({[Tags], [Tags_7], [Tags_8], [Tags_9], [Tags_10], [Tags_11], [Tags_12], [Tags_13], [Tags_14], [Tags_15], [Tags_16], [Tags_17], [Tags_18], [Tags_19], [Tags_20], [Tags_21], [Tags_22], [Tags_23]}, "tag1") then "tag1"
else if List.Contains({[Tags], [Tags_7], [Tags_8], [Tags_9], [Tags_10], [Tags_11], [Tags_12], [Tags_13], [Tags_14], [Tags_15], [Tags_16], [Tags_17], [Tags_18], [Tags_19], [Tags_20], [Tags_21], [Tags_22], [Tags_23]}, "tag2") then "tag2"
else if List.Contains({[Tags], [Tags_7], [Tags_8], [Tags_9], [Tags_10], [Tags_11], [Tags_12], [Tags_13], [Tags_14], [Tags_15], [Tags_16], [Tags_17], [Tags_18], [Tags_19], [Tags_20], [Tags_21], [Tags_22], [Tags_23]}, "tag3") then "tag3"
else "zzzNone"),
RemoveDummy = Table.SelectRows(Tags, each [ID] <> "ID-1234"),
#"Grouped Rows" = Table.Group(RemoveDummy, {"Max Series", "Tags"}, {{"Points per Tags", each List.Sum([#"Custom field (Points)]), type number}}),
#"Sorted Rows" = Table.Sort(#"Grouped Rows",{{"Tags", Order.Ascending}}),
#"Pivoted Column" = Table.Pivot(#"Sorted Rows", List.Distinct(#"Sorted Rows"[Tags]), "Tags", "Points per Tags"),
#"Renamed Columns" = Table.RenameColumns(#"Pivoted Column",{{"zzzNone", "None"}, {"Max Series", "Series"}}),
#"Added Custom1" = Table.AddColumn(#"Renamed Columns", "Last update", each DateTime.LocalNow()),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Custom1",{{"Last update", type datetime}})
in
#"Changed Type1"
The "Series" and "Tags" columns are a multivariable field, containing all series and tags and is translated by excel into multiple columns. The issue is that the number of series and tags are changing and to try coping with this I have created a dummy row with a lot of series. However, as you can see from the code this also changes and somehow "Tags_2" to "Tags_6" has disappeared and I had to error correct by removing these from the code.
Is there a dynamic way to if any column "Tags_*" contains "tag1" then... so I don't have to hard-code this?
Same goes for the "Max Series" where I would like to dynamically take max value of any columns "Series_*"
I would like to make the “Tags” step more dynamic, so that I can take input from a table in the excel sheet specifying which tags I want to search for instead of hardcoding “tag1, “tag2” etc.
My current code only assigns the points to the first tag found. However, I would like to assign points to several tags, so if two tags were found the "points" be assigned with half to each and for 3 tags they would all get one third of the points. I don’t know how to do this. Could you help me here?
As I am a bit new powerquery my code might be far from optimal, if you have some suggestions in your answers on how I can improve it that would be highly appreciated :-)
Hi mitru and welcome to StackOverflow!
You can make the 'Tags' step automatic by 'Unpivot other columns' and 'Group by' operations. To obtain this you should select all non Tag* columns and use 'Unpivot other columns'. Then please perform a Group by operation with operation = All Rows
You will receive a column populated with tables. The next step is to create a Custom columns with following formula:
=if List.Contains([Tags][Value],"tag1") then "tag1"
else if List.Contains([Tags][Value],"tag2") then "tag2"
else if List.Contains([Tags][Value],"tag3") then "tag3"
else "zzzNone"
The [Tags] is the column containing tables while [Value] is the column within each table that contains tags you are looking for.
Under below link there is a file with sample solution that I created.
https://sendeyo.com/en/608c8dee7f
Regarding bullet 3. I am not sure how the scoring system should work. Can you provide a sample data with the final output?
With the help from Gonso's post I was able to make the "tags" step more dynamic.
Furthermore, I found a solution on the bullet 3 assigning points to different tags if more than one tag is present.
I am posting the updated code here in case anyone else find the solution helpful:
let
Source = Csv.Document(Web.Contents("somefile"),[Delimiter=",", Columns=50, Encoding=65001, QuoteStyle=QuoteStyle.None]),
Promoted_Headers = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
Changed_Type_with_Locale = Table.TransformColumnTypes(Promoted_Headers, {{"Custom field (Points)", type number}}, "en-GB"),
Max_Series = Table.AddColumn(Changed_Type_with_Locale, "Max Series", each List.Max({[Series], [Series_1], [Series_2], [Series_3], [Series_4]})),
Unpivoted_Other_Columns = Table.UnpivotOtherColumns(Max_Series, {"Type", "ID", "Custom field (Points)", "Series", "Series_1", "Series_2", "Series_3", "Series_4", "Series_5", "Series_6", "Series_7", "Series_8", "Series_9", "Max Series"}, "Attribute", "Value"),
Tags = Table.Group(Unpivoted_Other_Columns, {"Type", "ID", "Custom field (Points)", "Series", "Series_1", "Series_2", "Series_3", "Series_4", "Series_5", "Series_6", "Series_7", "Series_8", "Series_9", "Max Series"}, {{"Tags", each _, type table [Type=nullable text, ID=nullable text, #"Custom field (Points)"=nullable number, Series=nullable text, Series_1=nullable text, Series_2=nullable text, Series_3=nullable text, Series_4=nullable text, Series_5=nullable text, Series_6=nullable text, Series_7=nullable text, Series_8=nullable text, Series_9=nullable text, Max Series=text, Attribute=text, Value=text]}}),
Tag1 = Table.AddColumn(Tags, "Tag1", each if List.Contains([Tags][Value],"Tag1") then 1
else 0),
Tag2 = Table.AddColumn(Tag, "Tag2", each if List.Contains([Tags][Value],"tag2") then 1
else 0),
Tag3 = Table.AddColumn(Tag2, "Tag3", each if List.Contains([Tags][Value],"tag3") then 1
else 0),
NoneTag = Table.AddColumn(Tag3, "TagNone", each if List.Sum({[Tag], [Tag2], [Tag3]}) > 0
then 0
else 1),
Tag_total = Table.AddColumn(NoneTag, "Tag_total", each List.Sum({[Tag], [Tag2], [Tag3], [TagNone]})),
Tag_update = Table.ReplaceValue(Tag_total,each [Tag1], each if [Tag1] > 0 then ([#"Custom field (Points)"] * ([Tag1] / [Tag_total])) else [Tag1],Replacer.ReplaceValue,{"Tag1"}),
Tag2_update = Table.ReplaceValue(Tag_update, each [Tag2], each if [Tag2] > 0 then ([#"Custom field (Points)"] * ([Tag2] / [Tag_total])) else [Tag2],Replacer.ReplaceValue,{"Tag2"}),
Tag3_update = Table.ReplaceValue(Tag2_update, each [Tag3], each if [Tag3] > 0 then ([#"Custom field (Points)"] * ([Tag3] / [Tag_total])) else [Tag3],Replacer.ReplaceValue,{"Tag3"}),
TagNone_update = Table.ReplaceValue(Tag3_update, each [TagNone], each if [TagNone] > 0 then ([#"Custom field (Points)"] * ([TagNone] / [Tag_total])) else [TagNone],Replacer.ReplaceValue,{"TagNone"}),
RemoveDummy = Table.SelectRows(TagNone_update, each [ID] <> "ID-1234"),
Grouped_Series = Table.Group(RemoveDummy, {"Max Series"}, {{"Tag1", each List.Sum([Tag1]), type number}, {"Tag2", each List.Sum([Tag2]), type number}, {"Tag3", each List.Sum([Tag3]), type number}, {"None", each List.Sum([TagNone]), type nullable number}}),
Sorted_Series = Table.Sort(Grouped_Series,{{"Max Series", Order.Ascending}}),
Renamed_Series = Table.RenameColumns(Sorted_Series,{{"Max Series", "Series"}}),
Added_last_update = Table.AddColumn(Renamed_Series, "Last update", each dateTime.LocalNow()),
Changed_date_Type = Table.TransformColumnTypes(Added_last_update,{{"Last update", type datetime}})
in
Changed_date_Type
Power BI junior here
How to look in each excel file from a SharePoint list and extract contents from predefined cells.
I am currently accessing a few intranet Sharepoint libraries containing .xlsx files and with the metadata of those files I am doing some reporting. For example, a library contains 10 excel files so I can graph who uploaded them, when they were uploaded, and wat category they were assigned to...
However, is there a way with Power Query to look into each and every of the files, take the value from, say cell A1 of the excel, and add it as a new column "CellA1Content"? I.e., make your own metadata from the content of the files and add them to the imported metadata table.
I've found some functions that I possibly might need:
File.Contents
Excel.CurrentWorkbook
However I am not well-versed enough in Power Query to put it all together, if it's even possible at all. I would have to do a foreach operation of some kind.
Edit: Solution
This worked. I selected the first non-hidden sheet in the excel and I also made the function so that I can pass the column and row number.
Main query:
let
Source = SharePoint.Contents("http://mysharepoint", [Implementation=null, ApiVersion=15]),
... ... ...
//Open each excel and get cell D5
#"AddedColumn1" = Table.AddColumn(#"Filtered Rows", "AddedColumn1", each GetCellContent([Content],4,5))
in
AddedColumn1
Blank query in Power BI, called GetCellContent:
let
Source = (binaryParameter,col,row) => let
Source = Excel.Workbook(binaryParameter, null, false),
UnhiddenSheets = Table.SelectRows(Source, each if [Hidden]=false and [Kind]="Sheet" then true else false),
Sheet = UnhiddenSheets{0}[Data],
Column = Table.SelectColumns(Sheet,{Text.Combine({"Column",Number.ToText(col)})}),
Cell = Record.Field(Column{row-1}, Text.Combine({"Column",Number.ToText(col)}) )
in
Cell
in
Source
You'll need a Function used in a column like this.
This is my local interpretation of your problem, without sharepoint. The same logic is shared though.
Main Query
let
Source = Folder.Contents("YourDirectory"),
#"Filtered Rows" = Table.SelectRows(Source, each ([Extension] = ".xlsx")),
#"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"Content", "Name"}),
#"Added Custom" = Table.AddColumn(#"Removed Other Columns", "Row1Col1", each PullRow1Col1([Content]))
in
#"Added Custom"
PullRow1Col1:
let
Source = (binaryParameter) => let
Source = Excel.Workbook(binaryParameter, null, false),
Sheet1_sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
Column1 = Sheet1_sheet{0}[Column1]
in
Column1
in
Source
How do you figure out what workbook a query in Excel points to?
After much back and forth I managed to come up with the following sub, which produces a string I can extract the name and path of the workbook from, but I feel there ought to be a simpler way to find this info? Am I missing something.
The sub I've produced is
Sub test()
Dim wbq As WorkbookQuery
For Each wbq In ThisWorkbook.Queries
Debug.Print wbq.Formula
Next wbq
End Sub
which prints something like
let
Source = Excel.Workbook(File.Contents("[what I want]"), null, true),
Sheet2 = Source{[Name="Sheet1"]}[Data],
#"Removed Columns" = Table.RemoveColumns(Sheet2,{"Column2", "Column3", "Column4", "Column5", "Column6", "Column7", "Column8", "Column9", "Column10", "Column12", "Column13", "Column14", "Column15", "Column16", "Column17", "Column18"}),
#"Removed Top Rows" = Table.Skip(#"Removed Columns",4),
#"Promoted Headers" = Table.PromoteHeaders(#"Removed Top Rows", [PromoteAllScalars=true]),
#"Removed Blank Rows" = Table.SelectRows(#"Promoted Headers", each not List.IsEmpty(List.RemoveMatchingItems(Record.FieldValues(_), {"", null})))
in
#"Removed Blank Rows"
My understanding is there is no simpler way than what you've shown. The query is written in M/Power Query Formula language -- which, to my knowledge, can't be evaluated, or interfaced with, in VBA.
I think the best you can do is to get the WorkbookQuery.Formula (as you're currently doing) and resort to string matching to try to extract the path being passed.
This is fine when the path is hard coded within the call to Excel.Workbook(File.Contents("...")) (as in your example), but harder if indirection is involved and the function is being passed an identifier/variable -- as you end up trying to re-create parts of the language's evaluation engine.
But maybe someone else can tell you differently.
I'm having multiple PowerQuery queries that I would like to feed the value of a cell in my Excel file. In this particular case, the full path to the sourcefile name.
Is there any way I can get this into PowerQuery?
This can be achieved using a named range and a custom function in PowerQuery:
Name the cell you need to refer (type in a name into the file left of the formula bar) - e.g. SourceFile
Insert a new blank PowerQuery query (PowerQuery ribbon -> From other sources)
In the PowerQuery editor, go to View -> Advanced Editor and paste the following code;
(rangeName) =>
Excel.CurrentWorkbook(){[Name=rangeName]}[Content]{0}[Column1]
Name the query to GetValue (Name property in the Query settings pane on the right)
Now you can access the named cell in your queries, using GetValue(cellName) - e.g.
= Excel.Workbook(File.Contents(GetValue("SourceFile")))
If the cell is part of an Excel table, the above is not needed - you can import/access that table's data directly using the "From Table/Range" button in the "Data" ribbon.
I couldn't get the => syntax to work so what I ended up with is as below. Here I'm using a single table to store all the values but you could also use named ranges and stick to using [content]{0}[Column1] for each one.
let
SITE_URL_VALUE = Excel.CurrentWorkbook(){[Name="SETTINGS_TABLE"]}[Content]{0}[Value],
FOLDER_PATH_VALUE = Excel.CurrentWorkbook(){[Name="SETTINGS_TABLE"]}[Content]{1}[Value],
FILENAME_VALUE = Excel.CurrentWorkbook(){[Name="SETTINGS_TABLE"]}[Content]{2}[Value],
Source = SharePoint.Files(SITE_URL_VALUE, [ApiVersion = 15]),
#"Import Filename" = Source{[Name=FILENAME_VALUE, #"Folder Path"=FOLDER_PATH_VALUE]}[Content],
#"Import Workbook" = Excel.Workbook(#"Import Filename"),
#"Import Table" = #"Import Workbook"{[Item="ACTIVITY_PLAN_TABLE",Kind="Table"]}[Data],
#"Removed Other Columns" = Table.SelectColumns(#"Import Table",{"Work Order", "MAT", "Exp Hours", "OSC", "Team", "Plant", "Area", "Hours", "Complete", "Plan Status"}),
#"Changed Type" = Table.TransformColumnTypes(#"Removed Other Columns",{{"Work Order", type text}, {"MAT", type text}})
in
#"Changed Type"