What are the differences between an item of type #table and a list of type #record in powerQuery? For example:
data = {
[id=1, name="tom"],
[id=2, name="sarah]
}
And:
data = #table(
{"id", "name"},
{
{1, "tom"},
{2, "sarah"}
},
)
Are they two ways to write the same thing, or should one be used over the other in certain cases?
The main difference is table may require strict data types, and only includes records, while list of records can also include values of other types, such as numbers or characters.
Your example can have column types defined in advance:
data = #table(
type table [id=Int64.Type, name=Text.Type],
{
{1, "tom"},
{2, "sarah"}
},
)
As opposite to a list:
data = {
[id=1, name="tom"],
[id=2, name="sarah"],
1,
"a"
}
Related
This question already has answers here:
How to slice a struct array?
(4 answers)
Closed 2 years ago.
How can I get all values of a complete column (fieldname), for all rows, from an Octave struct?
I would get it into a cell array, or a regular vector, preferably without looping.
You seem to be confusing a few things. Partly because of your equivalence comparison of structs to "R dataframes / python pandas".
Structs are better thought of as being similar to python dicts, R lists, etc. They are a special object that can hold 'fields', which can be accessed by a 'fieldname' ( or values accessed by keys, if you prefer ).
Also, like any other object in octave, they are valid elements for an array. This means you can have something like this:
octave:1> struct( 'name', { 'Tom', 'Jim'; 'Ann', 'Sue' }, 'age', { 20, 21; 22, 23 } )
S =
2x2 struct array containing the fields:
name
age
In general, when one deals with such a struct array, accessing a field on more than one elements of the array, produces a comma separated list. E.g.
octave:6> S(2,:).name
ans = Ann
ans = Sue
This can be passed to (i.e. "expanded into") any function that expects such a comma separated list as arguments. E.g.
octave:7> fprintf( 'The girls names are %s, and %s.\n', S(2,:).name )
The girls names are Ann, and Sue.
If you want, you can also pass that list straight into a 'cell constructor', to create a cell. (though if you want it to have a particular shape, you'll have to reshape it afterwords). E.g.
octave:9> reshape( { S.age }, size(S) )
ans =
{
[1,1] = 20
[2,1] = 22
[1,2] = 21
[2,2] = 23
}
There is also struct2cell but this does something different. Try it to see what it does (e.g. C = struct2cell(S) ).
Finally, to avoid confusion, given the fact that when one deals with struct arrays, "columns" refer to columns in the 'array', I would avoid referring to "fieldnames" by that term.
I have a bill of materials represented in ArangoDB as a directed acyclic graph. The quantity of each part in the bill of materials is represented on the edges while the part names are represented by the keys of the nodes. I'd like to write a query which traverses down the DAG from an ancestor node and sums the quantities of each part by its part name. For example, consider the following graph:
Qty: 2 Qty: 1
Widget +------> Gadget +------> Stuff
+ + Qty: 4
| Qty: 1 +---------> Thing
+----------------------------^
Widget contains two Gadgets, which each contains one Stuff and four Things. Widget also contains one Thing. Thus I'd like to write an AQL query which traverses the graph starting at widget and returns:
{
"Gadget": 2,
"Stuff": 2,
"Thing": 9
}
I believe collect aggregate may be my friend here, but I haven't quite found the right incantation yet. Part of the challenge is that all descendant quantities of a part need to be multiplied by their parent quantities. What might such a query look like that efficiently performs this summation on DAGs of depths around 10 layers?
Three possible options come to mind:
1.- return the values from the path and then summarize the data in the app server:
FOR v,e,p IN 1..2 OUTBOUND 'test/4719491'
testRel
RETURN {v:v.name, p:p.edges[*].qty}
This returns Gadget 2, Stuff [2,1], Thing [2,4], Thing [ 1 ]
2.- enumerate the edges on the path, to get the results directly :
FOR v,e,p IN 1..2 OUTBOUND 'test/4719491'
testRel
let e0 = p.edges[0].qty
let e1 = NOT_NULL(p.edges[1].qty,1)
collect itemName = v.name aggregate items = sum(e0 * e1)
Return {itemName: itemName, items: items}
This correctly returns Gadget 2, Stuff 2, Thing 9.
This obviously requires that you know the number of levels before hand.
3.- Write a custom function "multiply" similar to the existing "SUM" function so that you can multiply values of an array. The query would be similar to this :
let vals = (FOR v,e,p IN 1..2 OUTBOUND 'test/4719491'
testRel
RETURN {itemName:v.name, items:SUM(p.edges[*].qty)})
for val in vals
collect itemName = val.itemName Aggregate items = sum(val.items)
return {itemName: itemName, items: items}
So your function would replace the SUM in the inner sub-select. Here is the documentation on custom functions
I have been trying to get this dynamic query to work with dates as shown below in ArangoDB 3.1.
This works perfectly when I'm not trying to query dates, but returns an empty list as soon as I try to query with a date like below...
{
query:
'For c IN ##collectionName
FILTER ( c.#p0 == #v0 AND c.#p1 >= #v1 AND c.#p2 <= #v2 )
LIMIT #count RETURN c ',
bindVars: {
'#collectionName': 'Event',
p0: 'isPublished',
v0: true,
p1: 'dates[*].startsAt',
v1: '2018-06-01T04:00:00.000Z',
p2: 'dates[*].startsAt',
v2: '2018-07-01T03:59:59.999Z',
count: 9
}
}
Need some help getting past this
There are mistakes in your query, but they are actually not related to dates:
dates[*].startsAt is not a valid attribute path, but a shorthand expression for FOR date IN dates RETURN date.startsAt, which returns an array
The comparison operator >= does not work on arrays as you may think. null, true, false and every number and string are less than any array, see Type and Value Order. Your timestamp array will always be greater than any given timestamp string. What you probably want instead is an array comparison operator like ALL >=.
An expression dates[*].startsAt can not be used as bind parameter. With a document structure without array like { "date": { "startsAt": "..." } } it would be perfectly fine to bind ["date", "startsAt"] as p1 or p2. Note how the bind parameter value is an array of strings. "date.startsAt" on the other hand would describe the path for a top-level attribute
{ "date.startsAt": ... } and not a nested attribute startsAt of the top-level attribute date like { "date": { "startsAt": ... } }.
What your do with dates[*].startsAt is describing a top-level attribute like
{ "dates[*].startsAt": ... }, which does not exist. ["dates[*]", "startsAt"] does not work either. If you want to use the array expansion expression, then you have to write it like c.#p1a[*].#p1b in your query and use the bind parameters { "p1a": "dates", "p2a": "startsAt" }.
Query:
FOR c IN ##collectionName
FILTER c.#p0 == #v0
FILTER c.#p1a[*].#p1b ALL >= #v1
FILTER c.#p2a[*].#p2b ALL < #v2
LIMIT #count
RETURN c
bindVars:
{
"#collectionName": "Event",
"p0": "isPublished",
"v0": true,
"p1a": "dates",
"p1b": "startsAt",
"v1": "2018-06-01T04:00:00.000Z",
"p2a": "dates",
"p2b": "startsAt",
"v2": "2018-07-01T04:00:00.000Z",
"count": 9
}
I've read through the language specification of Power Query's M" language, and came across the open record type. My understanding is that the open type allows for other fields, but I don't have a concrete understanding of what that means.
The way to declare a normal (closed) record is simply
myRecord = [name = "MyName", Age = 30]
From the language specification (5.4: Record Types):
myRecordType1 = type [Name = text, Age = number] // Closed Record _type_
myRecordType2 = type [Name = text, Age = number, ...] // Open Record _type_
however,
myRecord = [Name = "MyName", Age = 30, ...] // Not valid code
So it seems this concept is only about custom record types, rather than records in general, but I have no idea what to do with it. I tried this:
testFunc = (inputArg as myRecordType2) => 1 // Not valid code
... expecting it might make the function only accept a record with Name & Age Fields, and optional other fields, but no. Thought it might not work with the as keyword, but even this doesn't work:
testTable = Table.AddColumn(Table.FromRecords({[A=1]}), "newcol", each [Name="MyName", Age=30], type myRecordType1) // Not valid code
Could someone illustrate a use(case) for this?
Did I miss something in the language spec?
My interpretation is as follows. Any comment is appreciated (even other thoughts).
Types are classifications of values. There are 2 flavors: primitive types (number, text etc.) and custom types, e.g. a specific table type or a record type.
For instance a table type is the set of column names, - types and any key values.
A table type can be specified first and then used when creating a table:
Tabeltype = type table[Key = number, Value = text],
TabletypeWithKey = Type.AddTableKey(Tabeltype,{"Key"},true),
TableWithKey = #table(TabletypeWithKey, {{1, "A"},{2, "B"}, {3, "C"}})
Likewise, you can create record types.
However, you can’t directly use a record type when creating a record.
You can use Value.ReplaceType to “ascribe” a type to a value, e.g. a record type to a record, provided the record type is closed and has no optional fields.
Example in the code below.
I would expect a possibility to verify if a value matches a specific type, but that can only be done with primitive types (using keyword “is” or “as” or “Type.Is”).
So I created the code below to check for conformity of records to record types, according to my interpretation: I can’t give any guarantees it’s full proof.
Currently it’s a query so you can see what’s happening, but you can easily turn it into a function and use examples in the bottom half of the code that’s currently commented out.
// Mext 2 lines to be decommented to turn the code into a function
//let
// fnCheckConformity = (Record1 as record, RecordType as type) as logical =>
let
// Next 2 lines to be removed when turning this code into a function
Record1 = [x = 1, A = 3, B = 4],
RecordType = type [x = number, optional y = text,...],
RecordTypeFields = Type.RecordFields(RecordType),
ToTable = Record.ToTable(RecordTypeFields),
RecordTypeTable = Table.ExpandRecordColumn(ToTable, "Value", {"Optional", "Type"}, {"Optional", "Type"}),
RecordTable = Table.FromColumns({Record.FieldNames(Record1),Record.FieldValues(Record1)},{"Record FieldName", "Record FieldValue"}),
JoinedTable = Table.Join(RecordTypeTable, "Name", RecordTable, "Record FieldName", JoinKind.FullOuter),
ConformityCheck = Table.AddColumn(JoinedTable, "Conform",
each if [Optional] = null then Type.IsOpenRecord(RecordType) else
if [Optional] then true else
if [Record FieldValue] <> null then Value.Is([Record FieldValue], [Type]) else
false),
Result = List.AllTrue(ConformityCheck[Conform])
in
Result
// Add a comma after Result when turning the code above into a function
/* The code below can be used when turning the code above into a function.
// Examples:
OpenRecordType = type [x = number, optional y = text,...],
ClosedRecordType = type [x = number, y = text],
RecordA = [x = 1],
RecordB = [x = 1, A = 3, B = 4],
RecordC = [x = 1, y = "MarcelBeug"],
// RecordC is ascribed type ClosedRecordType:
RecordCTyped = Value.ReplaceType(RecordC, ClosedRecordType),
Conformity1 = fnCheckConformity(RecordA, OpenRecordType), // true
Conformity2 = fnCheckConformity(RecordA, ClosedRecordType), // false
Conformity3 = fnCheckConformity(RecordB, OpenRecordType), // true
Conformity4 = fnCheckConformity(RecordB, ClosedRecordType), // false
Conformity5 = fnCheckConformity(RecordC, OpenRecordType) // true
in
Conformity5 */
Marcel's answer is great, but I could add a little more context.
It's true, there aren't a lot of uses for open record types in "M" today.
One place where it could have been useful is if we had the concept of "ragged" tables, e.g. this CSV with two, three, and four fields of data in the different rows.
A,B,C
1,2
1,2,3
1,2,3,4
Loading this CSV into PQ editor / Excel / PowerBI Desktop / powerbi.com will probably work, but it's not a good fit for a table value. In the "M" design today, a table is basically a list of closed records with non-optional fields (so you can't have a table row with more or less fields than the table columns).
Some other data sources like Azure Table or OData also could have used ragged tables. Right now, we'll return a table with some fixed columns, and a record column [Content] or [Open Types].
I see in the API docs there is a sort() method on List, but I'm not clear what it needs for a parameter. The current need is for a very simple straight up alpha comparison.
1. A Quick Solution
Thanks for the question! You can sort a list of Strings like this:
main() {
final List<String> fruits = <String>['bananas', 'apples', 'oranges'];
fruits.sort();
print(fruits);
}
The above code prints:
[apples, bananas, oranges]
2. Slightly more advanced usage
Notice that sort() does not return a value. It sorts the list without creating a new list. If you want to sort and print in the same line, you can use method cascades:
print(fruits..sort());
For more control, you can define your own comparison logic. Here is an example of sorting the fruits based on price.
main() {
final List<String> fruits = <String>['bananas', 'apples', 'oranges'];
fruits.sort((a, b) => getPrice(a).compareTo(getPrice(b)));
print(fruits);
}
Let's see what's going on here.
A List has a sort method, which has one optional parameter: a Comparator. A Comparator is a typedef or function alias. In this case, it's an alias for a function that looks like:
int Comparator(T a, T b)
From the docs:
A Comparator function represents such a total ordering by returning a negative integer if a is smaller than b, zero if a is equal to b, and a positive integer if a is greater than b.
3. How to do it with a list of custom objects
Additionally, if you create a list composed of custom objects, you could add the Comparable<T> as a mixin or as inheritance (extends) and then override the compareTo method, in order to recreate the standard behavior of sort() for your list of custom objects. For more info, do check out this other, related StackOverflow answer.
Here is the one line code to achieve it.
fruits.sort((String a, String b)=>a.compareTo(b)); //fruits is of type List<String>
For Sorting Simple List of Integers or Strings:
var list = [5 , -5 ,1];
list.sort(); //-5 , 1 , 5
For Reversing the list order:
list.reversed;
For Sorting List of Objects or Map by field of it:
List<Map<String, dynamic>> list= [
{"name": "Shoes", "price": 100},
{"name": "Pants", "price": 50},
];
// from low to high according to price
list.sort((a, b) => a["price"].compareTo(b["price"]));
// from high to low according to price
list.sort((a, b) => b["price"].compareTo(a["price"]));
To add just one point to Seth's detailed answer, in general, in
(a, b) => foo(a, b)
passed into sort, the function foo should answer an integer result as follows:
if a < b, result should be < 0,
if a = b, result should be = 0, and
if a > b, result should be > 0.
For the above law of trichotomy to hold, both a and b must be Comparables.
use compareAsciiUpperCase instead of compareTo, as it supports strings and automatically ignores case sensitive:
import "package:collection/collection.dart";
data.sort((a, b) {
return compareAsciiUpperCase(a.name, b.name);
});
After today, you should just be able to do list.sort() .
The sort method's argument is now optional, and it defaults to a function that calls compareTo on the elements themselves. Since String is Comparable, it should Just Work now.
How I have solved this problem.
List<Product> _dataSavingListProducts = [];
List<Product> _dataSavingListFavoritesProducts = [];
void _orderDataSavingLists() {
_dataSavingListProducts.toList().reversed;
_dataSavingListFavoritesProducts.toList().reversed;
}