I have a structurised json file, containing a list of elements.
I would like to convert all of the elements (which have also a structure) to table format. I can manually do it from PowerQuery GUI for one element; example code below.
let
Source=
Json.Document(Web.Contents("<source_address>")),
elements = Source[elements],
elements1 = elements{0},
#"Converted into table" = Record.ToTable(elements1)
in
#"Converted into table"
I'd like to iterate through all elements (so from elements{0} to elements{x}) and keep them in one excel output table. All elements have the same structure (columns)
Would be useful to see some sample JSON, as it's difficult to say with certainty without looking at its structure. I've just used some dummy JSON string, which I'm assuming resembles yours (in terms of structure).
let
//Source = Json.Document(Web.Contents("<source_address>")),
Source = Json.Document("{""elements"": [{""someKey1"": ""someValue1""}, {""someKey2"": ""someValue2""}], ""someOtherKey"": ""cat""}"),
elements = Source[elements],
transformList = List.Transform(elements, Record.ToTable),
appended = Table.Combine(transformList)
in
appended
If you copy-paste the above to the Advanced Editor, replace the second Source = ... with the one from your original code, you can then check if it gives you what you're after.
Edit:
I think given the URL you've provided, the code below will save you pivoting the table later and give you desired output.
let
Source = Json.Document(Web.Contents("https://fantasy.premierleague.com/drf/bootstrap-static")),
elements = Source[elements],
toTable = Table.FromRecords(elements)
in
toTable
Which gives me the below:
Related
I had a problem spliting data in rows, used the solution provided by horseyride in the following link
Splitting rows into separate rows on all columns in Power Query.
Basically I am loocking to separete a row as breaks there are.
Many thanks #horseyride. The solution works in a simular problem. However, it's poping up the following error:
Expression.Error: We cannot convert a value of type Table to type Text.
Details:
Value=[Table]
Type=[Type]
My table is this one:
let
Source = Pdf.Tables(File.Contents("C:\Users\gmall\OneDrive\EF personales\EF\Temporales\IBK_Sueldo_PEN.pdf"), [Implementation="1.3"]),
Table002 = Source{[Id="Table002"]}[Data],
TableTransform = Table.Combine(List.Transform(List.Transform(Table.ToRecords(Source),
(x) => List.Transform(Record.ToList(x),each Text.Split(_,"#(lf)"))),
each Table.FromColumns(_,Table.ColumnNames(Source))))
in
TableTransform
Please let me know how to solve this issue:
Expression.Error: We cannot convert a value of type Table to type Text.
Details:
Value=[Table]
Type=[Type]
You need to use Table002 in step3 since that is the prior step name, not Source, which was the prior step name in my other answer
let
Source = Pdf.Tables(File.Contents("C:\Users\gmall\OneDrive\EF personales\EF\Temporales\IBK_Sueldo_PEN.pdf"), [Implementation="1.3"]),
Table002 = Source{[Id="Table002"]}[Data],
TableTransform = Table.Combine(List.Transform(List.Transform(Table.ToRecords(Table002),
(x) => List.Transform(Record.ToList(x),each Text.Split(_,"#(lf)"))),
each Table.FromColumns(_,Table.ColumnNames(Table002))))
in
TableTransform
I am trying to get some variables and numbers out from an Excel table using Matlab.
The variables below named "diffZ_trial1-4" should be calculated by the difference between two columns (between "start" and "finish"). However I get the error:
Undefined operator '-' for input arguments of type"
'cell'.
And I have read somewhere that it could be related to the fact that I get {} output instead of [] and maybe I need to use cell2mat or convert the output somehow. But I must have done that wrongly, as it did not work!
Question: How can I calculate the difference between two columns below?
clear all, close all
[num,txt,raw] = xlsread('test.xlsx');
start = find(strcmp(raw,'HNO'));
finish = find(strcmp(raw,'End Trial: '));
%%% TIMELINE EACH TRIAL
time_trial1 = raw(start(1):finish(1),8);
time_trial2 = raw(start(2):finish(2),8);
time_trial3 = raw(start(3):finish(3),8);
time_trial4 = raw(start(4):finish(4),8);
%%%MOVEMENT EACH TRIAL
diffZ_trial1 = raw(start(1):finish(1),17)-raw(start(1):finish(1),11);
diffZ_trial2 = raw(start(2):finish(2),17)-raw(start(2):finish(2),11);
diffZ_trial3 = raw(start(3):finish(3),17)-raw(start(3):finish(3),11);
diffZ_trial4 = raw(start(4):finish(4),17)-raw(start(4):finish(4),11);
You are right, raw contains data of all types, including text (http://uk.mathworks.com/help/matlab/ref/xlsread.html#outputarg_raw). You should use num, which is a numeric matrix.
Alternatively, if you have an updated version of Matlab, you can try readtable (https://uk.mathworks.com/help/matlab/ref/readtable.html), which I think is more flexible. It creates a table from an excel file, containing both text and numbers.
An Excel table as data source may contain error values (#NA, #DIV/0), which could disturbe later some steps during the transformation process in Power Query.
Depending of the following steps, we may get no output but an error. So how to handle this cases?
I found two standard steps in Power Query to catch them:
Remove errors (UI: Home/Remove Rows/Remove Errors) -> all rows with an error will be removed
Replace error values (UI: Transform/Replace Errors) -> the columns have first to be selected for performing this operations.
The first possibility is not a solution for me, since I want to keep the rows and just replace the error values.
In my case, my data table will change over the time, means the column name may change (e.g. years), or new columns appear. So the second possibility is too static, since I do not want to change the script each time.
So I've tried to get a dynamic way to clean all columns, indepent from the column names (and number of columns). It replaces the errors by a null value.
let
Source = Excel.CurrentWorkbook(){[Name="Tabelle1"]}[Content],
//Remove errors of all columns of the data source. ColumnName doesn't play any role
Cols = Table.ColumnNames(Source),
ColumnListWithParameter = Table.FromColumns({Cols, List.Repeat({""}, List.Count(Cols))}, {"ColName" as text, "ErrorHandling" as text}),
ParameterList = Table.ToRows(ColumnListWithParameter ),
ReplaceErrorSource = Table.ReplaceErrorValues(Source, ParameterList)
in
ReplaceErrorSource
Here the different three queries messages, after I've added two new column (with errors) to the source:
If anybody has another solution to make this kind of data cleaning, please write your post here.
let
src = Excel.CurrentWorkbook(){[Name="Tabelle1"]}[Content],
cols = Table.ColumnNames(src),
replace = Table.ReplaceErrorValues(src, List.Transform(cols, each {_, "!"}))
in
replace
Just for novices like me in Power Query
"!" could be any string as substitute for error values. I initially thought it was a wild card.
List.Transform(cols, each {_, "!"}) generates the list of error handling by column for the main funcion:
Table.ReplaceErrorValues(table_with errors, {{col1,error_str1},{col2,error_str2},{},{}, ...,{coln,error_strn}})
Nice elegant solution, Sergei
In my code I'm trying to use load with entries from a cell, but it is not working. The portion of my code below produces a 3 dimensional array of strings. The strings represent the paths to file names.
for i = 1:Something
for j = 1:Something Different
for k = 1: Yet Something Something Different
DataPath{j,k,i} = 'F:\blah\blah\blah\fileijk %file changes based on i,j,and k
end
end
end
In the next part of the code I want to use load to open the files using the path names defined in the code above. I do this using the code below.
Dummy = DataPath{l,(k-1)*TSRRange+m};
Data = load(Dummy);
The idea is for Dummy to take the string content out of DataPath so I can use it in load. By doing this I thought that Dummy would be defined as a string and not a cell, but this is not the case. How do I pull the string out of DataPath so I can use it with load? Thanks.
I have to load the data this way because the data is located in multiple folders. I can post more of the code if needed, but it is complex.
Dummy is a cell because you assigned a 3D cell array but are accessing a 2D cell with Dummy = Datapath{1,(k-1)*TSRRange+m}
I don't believe that you can expect to access all cell elements I this way. Instead, use three indices just as you did when creating it.
For example I loaded a module, and there is a table in this module with name "Table1". In the main file I have a table which I want to be the exact same copy of "Table1".
So how can I do it, if I have only a name of that table.
When I am trying to do it like this
str = "Table1"
t = str
I obviously get a string instead of table, so how can I get a table content that table content? What I want is to able somehow make this line of code
t = 'Table1'
be equvalent to this one
t = Table1
If str is the name of a global variable, use _G[str] to get its value.
Tables in Lua are a very flexible and important datatype. So much, that even modules are tables. If you know, that there is a table by a given name in the module, and you have it's name in a variable, just use the [] operator to get the table:
tablename = 'Table1' -- you get this from somewhere, assuming it's not fixed
require 'mymodule'
t = mymodule[tablename]
However, this is not a very good approach, because it assumes that you "know" that the module contains a table by the given name. You can always design modules that will export the table by a given standard name (which does not change):
require 'mymodule'
t = mymodule.Table1 -- equivalent to mymodule['Table1']