I have the following query which gets historical data from coinmarketcap for a particular coin, in this case EOS.
let
Source = Web.Page(Web.Contents(**"https://coinmarketcap.com/currencies/eos/historical-data/?start=20130428&end=20180309"**)),
Data0 = Source{0}[Data],
#"Changed Type" = Table.TransformColumnTypes(Data0,{{"Date", type date}, {"Open", type number}, {"High", type number}, {"Low", type number}, {"Close", type number}, {"Volume", type number}, {"Market Cap", type number}}),
#"Sorted rows" = Table.Sort(#"Changed type",{{"Date", Order.Ascending}})
in
#"Sorted rows"
I used CONCATENATE to create a link which uses the today date(same format) as the end date so it updates every day. How do I reference the cell which contains the link in Web.Contents? I haven't used Power Query before and I would appreciate it if someone could walk me through the steps. I am really frustrated that such a simple and certainly very used operation doesn't have a ready made solution. Referencing a static web link is straightforward but absolutely useless.
Thanks very much in advance for any responses.
You can refer to a cell in an Excel sheet, using syntax like:
Excel.CurrentWorkbook(){[Name=MyRangeName]}[Content]{0}[Column1]
That will lookup the value in table/named range: MyRangeName, at the 1st row, 1st column.
Pass that expression as the 1st parameter for Web.Contents.
When EOS takes off, you owe me a lambo ...
Related
I am using power query in Excel. I have a date column with date starting in 1/31/2015 and ending 1/31/2022. I have the below line of code that works just fine.
...
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"Reading", type number}, {"EM_Date", type date}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [EM_Date] <= #date(2021, 12, 31)),
...
Now if I change the code just slightly like below It stops working and gives me an error. "Power Query Expression.Error: The Date value must contain the Date component. Details: 43831"
...
#"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"Reading", type number}, {"EM_Date", type date}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [EM_Date] >= #date(2021, 1, 31)),
...
How do I fix this error?
For those that are coming to this post for answers. What I found is that the Column must be set to Dates. In my case I was calculating dates and the source column was not set as dates.
Hope this helps you!
I am trying to create a forecast (single table) for departments to input their assumptions on spending in a single table. Instead of entering amounts for every single month, I would like the user to enter the amount, frequency, start date, and end date for each category. To illustrate, see below the table with some sample data.
This is the result in Power Query (or Power BI) I am trying to get, which is my understanding of how to be able to run date slicers and filters in a Power BI model when comparing against actuals.
If this can't be done with DAX and instead must be done in excel (through look up formulas), how would you structure the formula?
Here is a PQ example that creates what you show as your desired table given what you show as your input:
To use Power Query
Select some cell in your Data Table
Data => Get&Transform => from Table/Range
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to better understand the algorithm
M Code
let
Source = Excel.CurrentWorkbook(){[Name="Table9"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"G/L", Int64.Type}, {"Dimension", type text}, {"Description", type text},
{"Amount", Int64.Type}, {"Repeat Every", type text}, {"Start Date", type date}, {"End Date", type date}}),
//Last possible date as Today + 5 years (to end of month)
lastDt = Date.EndOfMonth(Date.AddYears(Date.From(DateTime.FixedLocalNow()),5)),
//Generate list of all possible dates for a given row using List.Generate function
allDates = Table.AddColumn(#"Changed Type", "allDates", each let
lastDate = List.Min({lastDt,[End Date]}),
intvl = {1,3,6}{List.PositionOf({"Monthly","Quarterly","Semi Annual"},[Repeat Every])}
in
List.Generate(
()=> [Start Date],
each _ <= lastDate,
each Date.EndOfMonth(Date.AddMonths(_,intvl)))),
//Remove unneeded columns and expand the list of dates
#"Removed Columns" = Table.RemoveColumns(allDates,{"Repeat Every", "Start Date", "End Date"}),
#"Expanded allDates" = Table.ExpandListColumn(#"Removed Columns", "allDates"),
//Sort to get desired output
// Date column MUST be sorted to ensure correct order when pivoted
// Other columns sorted alphanumerically, but could change the sort to reflect original order if preferred.
#"Sorted Rows" = Table.Sort(#"Expanded allDates",{
{"allDates", Order.Ascending},
{"G/L", Order.Ascending},
{"Dimension", Order.Ascending}}),
//Pivot the date column with no aggregation
#"Pivoted Column" = Table.Pivot(
Table.TransformColumnTypes(#"Sorted Rows", {
{"allDates", type text}}, "en-US"),
List.Distinct(Table.TransformColumnTypes(#"Sorted Rows", {{"allDates", type text}}, "en-US")[allDates]),
"allDates", "Amount")
in
#"Pivoted Column"
Original Data
Results
I have a rather large power query (35k rows but lots of columns) and I face a situation where I sometimes have a single sales order tying to multiple stock numbers where I get multiple planned dates associated with multiple factories. I need to apply the earliest planned date - along with its associated factory to all lines of that sales order. I have looked into merging the query to itself - and this does seem to be able to get to the minimum planned date. However that takes a VERY long time and I still can't seem to figure out how to also get the associated factory of that date and apply to all the rows of that sales order.
Here is some sample data:
What I need to do is apply the Planned Date and Factory information in red to all the other lines of the SO #. Is this something tht can be done in Power Query?
Here is some MCode you may be able to adapt to your real data.
It
uses the data you show as a starting point
Groups by SOP#
Determines the first date and associated factory
Re-expands the table (except for the original date/factory columns
let
Source = Excel.CurrentWorkbook(){[Name="Table5"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"SO#", Int64.Type}, {"Stock #", Int64.Type}, {"Planned Date", type date}, {"Factory", type text}}),
//Group by SO#
// then return minimum date and corresonding factory
#"Grouped Rows" = Table.Group(#"Changed Type", {"SO#"}, {
{"All", each _, type table [#"SO#"=nullable number, #"Stock #"=nullable number, Planned Date=nullable date, Factory=nullable text]},
{"Planned Date", each List.Min([Planned Date]), Date.Type},
{"Factory", (t)=> List.Range(t[Factory],List.PositionOf(t[Planned Date],List.Min(t[Planned Date])),1){0}, Text.Type}
}),
//Expand the table (except for the original date and factory columns
#"Expanded All" = Table.ExpandTableColumn(#"Grouped Rows", "All", {"Stock #"}, {"Stock #"})
in
#"Expanded All"
I have a sample truncated rent roll below (actual is hundreds of lines long with different combinations of lease codes, vacancies, moveouts). There is some randomness to the data as there are vacancies which make "moveIn, LeaseEnd, moveOut" fields blank and sometimes there are moveOuts.
I would like to pivot the leaseCodes to separate columns as seen in the second image.
Is it possible to do this in power query without custom code? My initial thoughts were to do fill down on all of the fields except lease code and amount and then pivot the leaseCode Column with the Amount Column. But as you can see, I won't be able to do this because the date columns are sometimes blank due to vacancies or filled with moveouts.
Would really appreciate anyone's help on best way to navigate this.
Thank you,
Create Table1 by selecting data and using Data...From Table/Range [x] including headers. Right click unit column and Fill...Down... Select [unit, leaseCode, Amount] columns, right-click RemoveOtherColumns. Use the drop-box next to LeaseCode column header and uncheck (null) to get rid of those rows. Select leaseCode column then Transform...Pivot Column... and for value column use Amount. Do File ... Close and Load To ..... Only Create Connection. You should get something similar to this code (edited to replace datetime with date). Make sure this query is named Table1
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"unit", Int64.Type}, {"floorplan", type any}, {"unitArea", Int64.Type}, {"resident", type text}, {"tenant", type text}, {"marketRent", Int64.Type}, {"leaseCode", type text}, {"Amount", Int64.Type}, {"residentDeposit", Int64.Type}, {"otherDeposit", Int64.Type}, {"moveIn", type date}, {"leaseEnd", type date}, {"moveOut", type date}, {"balance", type any}}),
#"Filled Down" = Table.FillDown(#"Changed Type",{"unit"}),
#"Removed Other Columns" = Table.SelectColumns(#"Filled Down",{"unit", "leaseCode", "Amount"}),
#"Filtered Rows" = Table.SelectRows(#"Removed Other Columns", each ([leaseCode] <> null)),
#"Pivoted Column" = Table.Pivot(#"Filtered Rows", List.Distinct(#"Filtered Rows"[leaseCode]), "leaseCode", "Amount", List.Sum)
in #"Pivoted Column"
Back in normal excel, show Queries dialog box (if not already open) with Data...Queries and Connections... right click Table1 query and choose Duplicate. Delete every step after step 2 in the Applied Steps tab off to the right by right clicking step called "Filled Down" and using Delete until end. Use the drop-box next to [unit] column header and uncheck (null) to get rid of those rows. Select the [leaseCode, Amount] columns and right click Remove Columns to get rid of those two. Home...Merge Queries... and in the bottom drop down choose Table1 we created before. Click unit column in top section and then unit column in bottom section to link them. Join kind is left outer, the default. Hit OK. Click the double arrow atop the new column, uncheck unit, uncheck use original column name as prefix, then hit OK. Resort columns as needed. Change types as needed Close and load to table. Should generate code similar to
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"unit", Int64.Type}, {"floorplan", type any}, {"unitArea", Int64.Type}, {"resident", type text}, {"tenant", type text}, {"marketRent", Int64.Type}, {"leaseCode", type text}, {"Amount", Int64.Type}, {"residentDeposit", Int64.Type}, {"otherDeposit", Int64.Type}, {"moveIn", type date}, {"leaseEnd", type date}, {"moveOut", type date}, {"balance", type any}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([unit] <> null)),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"leaseCode", "Amount"}),
#"Merged Queries" = Table.NestedJoin(#"Removed Columns",{"unit"},Table1,{"unit"},"Table1",JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries", "Table1", {"mta", "watr", "Total", "garg"}, {"mta", "watr", "Total", "garg"})
in #"Expanded Table1"
In the Query Editor, highlight the lease code and amount columns and then pivot them (from the Transform tab on the Query Editor ribbon).
I am trying to make a table more readable by making a report.
This table has 3 columns;
Staff
Task (Each staff could have 0-many tasks)
Status (Planned, Started, Finished)
The report would have Staff as the left most column, 3 status as column headings. The values should be task values and if there are many tasks it should be concatenated, say, with a carriage return.
I tried pivoting but it didn't work since the task values are texts. I tried Power Query and it displays errors for every cell where there are more than 1 task.
Is there a way to do this? ...without VBA please.
Thanks
I presume you know how to operate Power Query Editor so I will skip the part of how to add data to the editor.
In my solution, I used the following sample data stored in Table3.
Once added data the editor will automatically recognize all data as text.
My approach is to add a custom column to combine Staff and Status as below first:
Then I grouped the data by the custom column (Staff+Status) with some Advanced Coding. You can do a Group By first and then go to Advanced Editor to change the formula as below:
= Table.Group(#"Added Custom", {"Staff+Status"}, {{"Task", each Text.Combine([Task],"#(lf)"), type text}})
Which will give you the following look:
Then you can split the custom column back to Staff and Status separately:
Then you can pivot the Status column, set Task as the Values Column, and in the Advanced Options set Don't Aggregate as the Aggregate Value Function.
Then you pretty much finished here, and you can load the query to a worksheet which may look like the following where the carriage return seems not working.
In order to 'activate' carriage return (which is actually line feed), you need to select a cell that you want to see carriage return, click somewhere in the formula bar, and you will notice the carriage return is 'activated'.
Copy the format of that cell and paste to the rest of the table using Format Painter to get the following:
If you are unclear about the above step, please read this article: How to display Power Query results with line feed or carriage return
All done. Cheers :)
By the way here are the codes behind the scene for reference only:
let
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Staff", type text}, {"Task", type text}, {"Status", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Staff+Status", each [Staff]&"+"&[Status]),
#"Grouped Rows" = Table.Group(#"Added Custom", {"Staff+Status"}, {{"Task", each Text.Combine([Task],"#(lf)"), type text}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Grouped Rows", "Staff+Status", Splitter.SplitTextByDelimiter("+", QuoteStyle.Csv), {"Staff+Status.1", "Staff+Status.2"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Staff+Status.1", type text}, {"Staff+Status.2", type text}}),
#"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[#"Staff+Status.2"]), "Staff+Status.2", "Task"),
#"Renamed Columns" = Table.RenameColumns(#"Pivoted Column",{{"Staff+Status.1", "Staff"}})
in
#"Renamed Columns"