I am looking for a solution to what should be a simple issue. I come from a MySQL development and still have problems here and a with AQL. I want to display the comments of a user with page pagnation without duplicate article keys. I have a comment collection with
_key, user_key, content_key, comment, stamp_create
Example:
1, 4483, 200, "reply 1", "2021-04-24T14:55:55+00:00"
2, 4483, 200, "reply 2", "2021-04-24T14:56:23+00:00"
3, 4483, 201, "reply 1", "2021-04-23T17:10:15+00:00"
4, 4483, 202, "reply 1", "2021-04-23T12:30:35+00:00"
5, 4483 , 202, "reply 2", "2021-04-22T23:50:51+00:00"
Now I need the output all comments Sorted by stamp_create DESC but with summarized content_key, With Mysql a "Group By" was enough for this.
So I try it with Collect.
FOR c IN comments FILTER c.user_key =="4483" SORT c stamp_create DESC COLLECT contentKeys = c.content_key RETURN contentKeys LIMIT 0,3
Generally the result is correct, but the sorting doesn't work because COLLECT doesn't remember it. Well then with the good old DISTINCT:
FOR c IN comments FILTER c.user_key =="4483" SORT c stamp_create DESC RETURN DISTINCT contentKeys LIMIT 0,3
Result looks good and the sorting is correct, but I get only 2 results and not 3 as specified in Limit. For a page pagnation unfortunately useless, because I get more or less results (MySQL does it differently with Group By).
Next try now with a little more effort:
FOR c IN comments
FILTER c.user_key =="4483"
COLLECT contentKeys = c.content_key INTO groups
LET keys = first(FOR value IN groups[*].c SORT value.stamp_create DESC RETURN {key: value._key, stamp: value.stamp_create})
SORT keys.stamp DESC
LIMIT 0,3
RETURN keys
I load here all comments grouped in "groups", then sort them within the group by and then again in the whole.
The result is correct and I also get 3 results. Big disadvantage that is not really fast and I have many users with more than 60K comments.
The entire (filtered) data set (can also be 60K long) must be read in and then re-sorted. This must be easier to do.
So my question is, is there a performant way to make this faster and easier?
thanks
With the help of mpoeter, we found a faster and more elegant method.
Here is the solution:
FOR c IN comments
FILTER c.user_key =="4483"
COLLECT contentKeys = c.content_key AGGREGATE maxStamp = MAX(c.stamp_create) OPTIONS { method: "sorted" }
SORT maxStamp DESC
LIMIT 0,3
RETURN contentKeys
Thanks to the ArangoDB team :-)
I have below JSON output from an API, in Office Excel I am importing data via Web from API.
[{
"level": 1,
"children": [{
"level": 2,
"children": [{
"level": 3,
"name": "Chandni Chowk",
"data": ["Data 1", "Data 2"]
}],
"name": "Delhi",
"data": ["Delhi Area"]
}],
"name": "Country",
"data": ["India", "Bangladesh"]
}]
https://learn.microsoft.com/en-us/powerquery-m/quick-tour-of-the-power-query-m-formula-language
I have above document.
let
Source = Json.Document(Web.Contents("MY API URL GOES HERE")),
AsTable = Table.FromRecords(Source)
----
----
in
#"Renamed Column2"
In the power query editor I have this for now.
As a out put in Excel file I need like this.
Country Delhi Chandni Chowk
India Delhi Area Data 1
Bangladesh Data 2
Can I get this data from this JSON or I need to change my JSON output format which matches power query?
Power Query interprets JSON as a hierarchy of records and lists. My goal is to flatten the JSON into a record like this and then convert it into a table:
Country : {"India", "Bangladesh"}
Delhi : {"Delhi Area"}
Chandni Chowk : {"Data 1", "Data 2"}
At any particular level, we can pull the name and data value using Record.FromList:
Record.FromList({CurrentLevel[data]}, {CurrentLevel[name]})
For the first level, this is
Record.FromList({{"India","Bangladesh"}}, {"Country"})
which corresponds to the first field in the goal record.
At any level, we can navigate to the next level like this:
NextLevel = CurrentLevel[children]{0}
Using these to building blocks, we can now write a custom function Expand to flatten the record:
1 | (R as record) as record =>
2 | let
3 | ThisLevel = Record.FromList({R[data]}, {R[name]}),
4 | CombLevel = if Record.HasFields(R, {"children"})
5 | then Record.Combine({ThisLevel, #Expand(R[children]{0})})
6 | else ThisLevel
7 | in
8 | CombLevel
Line 1: The syntax for defining a function. It takes a record R and returns a record after doing some transformations.
Line 3: How to deal with the current level, as mentioned earlier.
Line 4: Check if the record has another level to expand down to.
Line 5: If it does, then Record.Combine the result of the current level with the result of the next level, where the result of the next level is calculated by navigating to the next level and recursively applying the function we're defining. With three levels this looks like:
Record.Combine({Level1, Record.Combine({Level2, Level3})})
Line 6: Recursion stops when there are no more levels to expand. No more combinations, just the last level is returned.
All that's left is to transform it into the shape we want. Here's what my query looks like using the Expand function we just defined:
let
Source = Json.Document( < JSON Source > ),
ExpandRecord = Expand(Source{0}),
ToTable = Table.FromColumns(
Record.FieldValues(ExpandRecord),
Record.FieldNames(ExpandRecord)
)
in
ToTable
This uses Record.FieldValues and Record.FieldName as arguments in Table.FromColumns.
The step after using the Expand custom function looks like this in the query editor if you select the first list cell:
The final result is what you asked for:
I have a collection called prodSampleNew with documents that have hierarchy levels as fields in arangodb:
{
prodId: 1,
LevelOne: "clothes",
LevelTwo: "pants",
LevelThree: "jeans",
... etc....
}
I want take the hierarchy levels and convert them into their own documents, so I can eventually build a proper graph with the hierarchy.
I was able to get this to extract the first level fo the hierarchy and put it in a new collection using the following:
for i IN [1]
let HierarchyList = (
For prod in prodSampleNew
COLLECT LevelOneUnique = prod.LevelOne
RETURN LevelOneUnique
)
FOR hierarchyLevel in HierarchyList
INSERT {"name": hierarchyLevel}
IN tmp
However, having to put a for I IN [1] at the top seems wrong and that there should be a better way.(yes I am fairly new to AQL)
Any pointers on a better way to do this would be appreciated
Not sure what you are trying to achieve exactly.
The FOR i IN [1] seems unnecessary however, so you could start your AQL query directly with the subquery to compute the distinct values from hierarchy level 1:
LET HierarchyList = (
FOR prod IN prodSampleNew
COLLECT LevelOneUnique = prod.LevelOne
RETURN LevelOneUnique
)
FOR hierarchyLevel IN HierarchyList
INSERT {"name": hierarchyLevel} IN tmp
The result should be the same.
If the question is more like "how can I get all distinct names of levels from all hierarchies", then you could use something like
LET HierarchyList = UNIQUE(FLATTEN(
FOR prod IN prodSampleNew
RETURN [ prod.LevelOne, prod.LevelTwo, prod.LevelThree ]
))
...
to produce an array with the unique names of the hierarchy levels for level 1-3.
Shouldn't this answer your question, please describe the desired result the query should produce.
Am trying to compute a query to filter out products based on filterable options, color, size, e.t.c.
Use case: Obtain all products that have a size 8 and red
Attempt 1: Using Intersection
FOR product IN products
FILTER product.options != null
FOR productOption IN product.options
FILTER productOption.option == "Size" AND LENGTH( INTERSECTION( productOption.value, ["8","14","16"] ) ) > 0
AND productOption.option == "Color" AND LENGTH( INTERSECTION( productOption.value, ["Red"] ) ) > 0
RETURN product
Attempt 2: Using Multiple Filters
FOR product IN products
FILTER product.options != null
FOR productOption IN product.options
FILTER productOption.option == "Size"
FOR productSizeOptionValue IN productOption.value
FILTER productSizeOptionValue IN ["8","10"]
FILTER productOption.option == "Color"
FOR productColorOptionValue IN productOption.value
FILTER productColorOptionValue IN ["Red"]
RETURN product
Am not sure if my thinking should be along the lines of first obtaining possible product candidates using LET than merging the results of each individual LET query. Seemed rather clumsy! :(
Attempt 1 will not work because it contains two AND-combined FILTER conditions on productOption.option, with different comparison values:
...
FILTER
productOption.option == "Size"
AND LENGTH(INTERSECTION(...))
AND productOption.option == "Color"
AND LENGTH(INTERSECTION(...))
...
This won't work, because the option value cannot be "Size" and "Color" at the same time.
I think it may be more sensible to model product options as separate attributes instead of using generic option arrays. This will work if a product can only have a single value for each distinct option but not multiple. In this approach, a product would look like this:
{ "Size": "8", "Color": "Red", "HasWheels": true }
Then filtering on product attributes become easier, e.g.
FOR product IN products
FILTER product.Size IN [ "8", "10" ]
AND product.Color == "Red"
RETURN product
Search conditions could then also be OR-combined, e.g.
FILTER product.Size IN [ "8", "10" ]
OR product.Color == "Red"
This approach will make querying much easier. It comes with one disadvantage though, and that is that if there are many different product attributes, you will not be able to index most of them due to memory constraint. This is not a problem if there are few different product attributes, or if all search conditions use a few common attributes that are selective enough that an index greatly speeds up the lookup.
I'm testing some Couchdb features and I want get results with a reversed order by insertion date, querying by "i" field
A sample doc:
{
"_id": "970c3a0fdbb23dde47fb4075091a4d2b",
"_rev": "1-54448147611ff5e89189bb44e58c1521",
"doc_type": "Test",
"e": "3/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/0/36/2",
"d": "64/183/329/2/360/10/13/47/6/351/331/320/355/342/7/335/47/18/30/56/18/323/351/325/323/218/163/155/155/155/155",
"f": "1399305161/1399305185/1399305194/1399305254/1399305314/1399305374/1399305434/1399305447/1399305506/1399305566/1399305626/1399305668/1399305727/1399305787/1399305847/1399305908/1399305963/1399305970/1399306022/1399306068/1399306078/1399306100/1399306159/1399306219/1399306279/1399306308/1399306321/1399306379/1399306439/1399306493/1399306506",
"i": "3566120224",
"dated": "1399305161",
"v": "0/5/6/32/63/63/51/16/35/60/0/10/64/31/64/48/14/31/6/55/60/50/0/0/21/5/34/0/0/0/0",
"date": "2014-05-05T15:52:42Z"
}
My view:
function(doc) {
if(doc.i && doc.date){
emit([doc.i,doc.date],1); // 1 to test only
}
}
I'm testing it with:
myview?startkey=["3566120224"]&endkey=["3566120224",{}]&reversed=true
But I'm getting the data with a date order not reversed
{"total_rows":545,"offset":508,"rows":[
{"id":"407ee687674b783601ce6d7da906515e","key":["3566120224","2014-05-05T14:11:01Z"],"value":1},
{"id":"407ee687674b783601ce6d7da9062b51","key":["3566120224","2014-05-05T14:15:21Z"],"value":1},
{"id":"407ee687674b783601ce6d7da905f4d9","key":["3566120224","2014-05-05T14:19:41Z"],"value":1},
{"id":"407ee687674b783601ce6d7da905b4e1","key":["3566120224","2014-05-05T14:24:01Z"],"value":1},
{"id":"407ee687674b783601ce6d7da905733c","key":["3566120224","2014-05-05T14:28:22Z"],"value":1},
{"id":"407ee687674b783601ce6d7da904e7ea","key":["3566120224","2014-05-05T14:32:42Z"],"value":1},
{"id":"407ee687674b783601ce6d7da9043709","key":["3566120224","2014-05-05T14:37:02Z"],"value":1},
{"id":"407ee687674b783601ce6d7da9039896","key":["3566120224","2014-05-05T14:41:22Z"],"value":1},
{"id":"407ee687674b783601ce6d7da90303be","key":["3566120224","2014-05-05T14:45:43Z"],"value":1},
{"id":"407ee687674b783601ce6d7da90239ae","key":["3566120224","2014-05-05T14:50:03Z"],"value":1},
{"id":"407ee687674b783601ce6d7da9018442","key":["3566120224","2014-05-05T14:54:23Z"],"value":1},
{"id":"407ee687674b783601ce6d7da90104f0","key":["3566120224","2014-05-05T14:58:43Z"],"value":1},
{"id":"407ee687674b783601ce6d7da9007b67","key":["3566120224","2014-05-05T15:03:04Z"],"value":1},
{"id":"90bb394f7a4a581ff4dc78bfaffff448","key":["3566120224","2014-05-05T15:07:24Z"],"value":1},
{"id":"90bb394f7a4a581ff4dc78bfafff368e","key":["3566120224","2014-05-05T15:11:44Z"],"value":1},
{"id":"90bb394f7a4a581ff4dc78bfaffe7e65","key":["3566120224","2014-05-05T15:16:05Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091f8e5c","key":["3566120224","2014-05-05T15:24:45Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091f6241","key":["3566120224","2014-05-05T15:29:05Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091f254a","key":["3566120224","2014-05-05T15:33:26Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091ed01b","key":["3566120224","2014-05-05T15:37:46Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091e5f42","key":["3566120224","2014-05-05T15:42:06Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091dd992","key":["3566120224","2014-05-05T15:46:26Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091d3853","key":["3566120224","2014-05-05T15:50:47Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091c9a3c","key":["3566120224","2014-05-05T15:55:07Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091bf465","key":["3566120224","2014-05-05T15:59:27Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091ba442","key":["3566120224","2014-05-05T16:03:47Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091ad482","key":["3566120224","2014-05-05T16:08:08Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb4075091a2130","key":["3566120224","2014-05-05T16:12:28Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb40750919a6ef","key":["3566120224","2014-05-05T16:16:48Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb407509192479","key":["3566120224","2014-05-05T16:21:08Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb40750918a977","key":["3566120224","2014-05-05T16:25:29Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb40750917b468","key":["3566120224","2014-05-05T16:29:49Z"],"value":1},
{"id":"970c3a0fdbb23dde47fb407509170583","key":["3566120224","2014-05-05T16:34:09Z"],"value":1}
]}
I have two times the same data(date & dated[ms date]), 1399305161 that is = 2014-05-05T15:52:42Z
thinking that I could order the results with a data type more easy to parse for couchdb, but didn't work using the dated field
Also I don't need the id field, how can exclude it from the results?
If you look here, you can see that reverse is not a supported query option. To reverse the data, you want to do:
myview?endkey=["3566120224"]&startkey=["3566120224",{}]&descending=false
If you don't want the ID field, you can just return the reduced value at the highest grouping level:
myview?endkey=["3566120224"]&startkey=["3566120224",{}]&descending=false&group_level=2
To remove the doc.i just write:
function(doc) {
if(doc.i && doc.date){
emit([doc.date],1); // just removed doc.i
}
}
To reverse the results:
You can either change the results so that you use the descending option. I'd think that is bad practice though as it depends on the client using this feature.
Assuming it's the default use of the view, I'd opt to returning the date as integer. Date.parse(doc.date).valueof() should do the trick of returning the epoch time as integer or double. If you then return this with a '-' (minus) the view will by default be sorted in descending order. This is assuming you don't need the formatted date in the key.