invalid argument type in call to function 'ATTRIBUTES()' in ArangoDB - arangodb

I have stored my Data in the give formate in AreangoDB, My collection name in DSP:
"data": {
"1": [ {"db_name": "DSP"}, {"rel": "2"} ],
"2": [ {"rel_name": "DataSource"}, {"tuple": "201"}, {"tuple": "202"}, {"tuple": "203"} ],
"201": [ {"src_id": "Pos201510070"}, {"src_name": "Postgres"}, {"password": "root"}, {"host": "localhost"}, {"created_date": "20151007"}, {"user_name": "postgres"}, {"src_type": "Structured"}, {"db_name": "postgres"}, {"port": "None"} ],
"202": [ {"src_id": "pos201510060"}, {"src_name": "Postgres"},{"password": "root"}, {"host": "localhost"}, {"created_date": "20151006"}, {"user_name": "postgres"}, {"src_type": "Structured"}, {"db_name": "DSP"}, {"port": "5432"} ],
"203": [ {"src_id": "pos201510060"}, {"src_name": "Postgres"}, {"password": "root"}, {"host": "localhost"}, {"created_date": "20151006"}, {"user_name": "postgres"},{"src_type": "Structured"},{"db_name": "maindb"},{"port": "5432"} ]
}
I am executing a query with the above data in the following format:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p)
FOR attribute IN attributes
LET key = ATTRIBUTES(attribute)[0]
LET value = attribute[key]
RETURN { subject: attribute, predicate: key, object: value }
When I submit my query to ArangoDB, it returns the response as:
Warnings:
[1542], 'invalid argument type in call to function 'ATTRIBUTES()''
[1542], 'invalid argument type in call to function 'ATTRIBUTES()''
[1542], 'invalid argument type in call to function 'ATTRIBUTES()''
[1542], 'invalid argument type in call to function 'ATTRIBUTES()''
Result:
[
{
"subject": "data",
"predicate": null,
"object": null
},
{
"subject": "_id",
"predicate": null,
"object": null
},
{
"subject": "_rev",
"predicate": null,
"object": null
},
{
"subject": "_key",
"predicate": null,
"object": null
}
]
Please tell me what is the problem with this query, and why the answer is like the above..I am working in ArangoDB-2.7.3-win64.
Thanks

Let me demonstrate how to construct such a complex query digging deep into nested data structures. I start out taking the outer parts of the query, to have an inner result:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p)
FOR attribute IN attributes
RETURN attribute
which gives me:
[
"data",
"_rev",
"_key",
"_id"
]
So lets dive deeper into the next layer. I guess you're only interested in the values present underneath data key right? so we pick p.data:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p.data)
FOR attribute IN attributes
RETURN attribute
which then gives me the keys for your next inner array:
[
"203",
"202",
"201",
"2",
"1"
]
We now explore what we find attached to these nodes:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p.data)
FOR oneAttribute IN attributes
LET keys = p.data[oneAttribute]
RETURN keys
Its Again an array, which we need to iterate into using a FOR loop over keys:
[
[
{
"src_id" : "pos201510060"
},
{
"src_name" : "Postgres"
}, ...
We add this additional FOR-loop:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p.data)
FOR oneAttribute IN attributes
LET keys = p.data[oneAttribute]
FOR key IN keys
RETURN key
we get the inner most objects:
[
{
"src_id" : "pos201510060"
},
{
"src_name" : "Postgres"
},
{
"password" : "root"
},
...
You wanted to use the ATTRIBUTES function, but the objects only have one member, so we can access [0]:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p.data)
FOR oneAttribute IN attributes
LET keys = p.data[oneAttribute]
FOR key IN keys
LET keyAttributes=ATTRIBUTES(key)
RETURN keyAttributes
Which gives us the object keys, one per inner most object:
[
[
"src_id"
],
[
"src_name"
],
We inspect whether we now get only the object keys of the inner most structure; we pick the variable names a little more clever than above:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p.data)
FOR oneAttribute IN attributes
LET pairs = p.data[oneAttribute]
FOR onePair IN pairs
LET pairKey=ATTRIBUTES(onePair)[0]
RETURN pairKey
YES:
[
"src_id",
"src_name",
"password",
"host",
...
So now its time to construct the results object as you wanted them:
FOR p IN NestedDSP
LET attributes = ATTRIBUTES(p.data)
FOR oneAttribute IN attributes
LET pairs = p.data[oneAttribute]
FOR onePair IN pairs
LET pairKey=ATTRIBUTES(onePair)[0]
RETURN { subject: oneAttribute, predicate: pairKey, object: onePair[pairKey] }
The subject is the number identifying the outermost item, the predicate is the object key, and the object is the value in it:
[
{
"subject" : "203",
"predicate" : "src_id",
"object" : "pos201510060"
},
{
"subject" : "203",
"predicate" : "src_name",
"object" : "Postgres"
}
Which is hopefully what you wanted?

Related

Remove empty Keys from JSON arrays using Groovy

I would like to remove the array SEO from the json when the keys "Description" and "Title" in the has no value.
json:
[
{
"SEO": [
{
"Description": "",
"Title": ""
}
],
"accesoires": [
"1167296"
],
"shortCode": "S-576",
"spareParts": [
"800236"
]
}]
I tried the below code but i'm not able to remove the array.
def Message processData(Message message) {
def body = message.getBody(String);
def json = new JsonSlurper().parseText(body)
json.each{
it.SEO.each{
if(!(it.findResults{k, v -> v?.size() > 0 && v[0]?.length() > 0 ? v[0] : null })){
json.remove("SEO")
} } }
def out= JsonOutput.toJson(json)
message.setBody(out)
return message}
To remove the array "SEO" from the JSON when the keys "Description" and "Title" have no value, you can use the following Groovy code:
def jsonString = '[{"SEO": [{"Description": "", "Title": ""}], "accesoires": ["1167296"], "shortCode": "S-576", "spareParts": ["800236"]}]'
def json = new JsonSlurper().parseText(jsonString)
for (item in json) {
if (!item.SEO[0].Description && !item.SEO[0].Title) {
item.remove('SEO')
}
}
println(JsonOutput.toJson(json))
This will first parse the JSON string into a list of maps using JsonSlurper. Then it iterates through each map in the list and checks if the "Description" and "Title" keys in the "SEO" array are empty. If they are, it removes the "SEO" array from the map using the remove() method. Finally, it prints the modified JSON using the JsonOutput.toJson() method.

Fetch key value from json data using terraform

I have a json data file from which I need to fetch only the id attribute value into a separate list variable. I tried for loop but not able to get the required data using terraform. Can someone tell how to fetch the id part ? your help is much appreacited.
code:
locals {
data = jsondecode(file("./data.json"))[*]
sections = [ for item in local.data : item ]
}
output "ids" {
value = [for a in local.sections[0]: a]
}
json file:
{
"c":[
{
"id":"6",
"key":"c",
"name":"s01"
}
],
"l":{
"id":"7",
"key":"l",
"name":"s02"
},
"m":{
"id":"8",
"key":"mp",
"name":"s03"
},
"n":{
"id":"5",
"key":"cn",
"name":"s04"
},
"od":"odk",
"s":{
"id":"9",
"key":"cs",
"name":"s05"
},
"ss":{
"id":"1",
"key":"ss",
"name":"s06"
},
"in":{
"id":"65",
"key":"cn",
"name":"s07"
},
"b":{
"id":"2",
"key":"cb",
"name":"s08"
}
}
Not sure if "od":"odk", is mistake or not, but you can do the following:
locals {
data = jsondecode(file("./data.json"))
}
output "ids" {
value = [for v in values(local.data): v.id if can(v.id)]
}

How can I return all nested objects using python?

I wrote an Elastic query which will check the condition (status="APPROVED") and Gets all approved_by objects.
This is my index (portfolio):
{
"settings": {},
"mappings": {
"portfolio": {
"properties": {
"status": {
"type": "keyword",
"normalizer": "lcase_ascii_normalizer"
},
"archived_at": {
"type": "date"
},
"approved_by": {
"id": "text",
"name":"text"
}
}
}
}
}
Currently I have 60 objects whose status are approved , so when i run the query it will show 60 objects,but in my case i am getting only one object(I debugged the code, total 60 objects are coming as expected, but still returning only single object), please help guys.
My query:
profiles = client.search(index='portfolio', doc_type='portfolio',
scroll='10m', size=1000,
body={
"query": {"match": {"status": "APPROVED"}}
})
sid = profiles['_scroll_id']
scroll_size = len(profiles['hits']['hits'])
while scroll_size > 0:
for info in profiles['hits']['hits']:
item = info['_source']
approved_by_obj = item.get('approved_by')
if approved_by_obj:
return (jsonify({"approved_by": approved_by_obj}))
Expected o/p format:
{
"approved_by": {
"id": "system",
"name": "system"
}
}
You're getting only one result because you're returning from your loop, effectively breaking out of it completely.
So, instead of returning from it, append the found approved_by_object to a list of your choice and then return that 60-member list:
profiles = client.search(index='portfolio', doc_type='portfolio',
scroll='10m', size=1000,
body={
"query": {"match": {"status": "APPROVED"}}
})
sid = profiles['_scroll_id']
scroll_size = len(profiles['hits']['hits'])
approved_hits_sources = [] # <-- add this
while scroll_size > 0:
for info in profiles['hits']['hits']:
item = info['_source']
approved_by_obj = item.get('approved_by')
if approved_by_obj:
approved_hits_sources.append({"approved_by": approved_by_obj}) # <--- append and not return
return jsonify({"approved_hits_sources": approved_hits_sources})

Power Query Expression error: A value of type "Record" cannot converted into type "Text"

I have the issue that I'm not able to execute the following code. The syntax seems to be okay, but when I try to execute it, I get the response, that:
Expression.Error: We cannot convert a value of type Record to type "Text".
Details:
Value=[Record]
Type=[Type]
let
body="{
""page"": ""1"",
""pageSize"": ""100"",
""requestParams"": {
""deviceUids"": [
""xxx-yyy-xxx-yyyy-xxxx"",
""yyy-xxx-yyy-xxxx-yyyy"",
""aaa-bbb-aaa-bbbb-aaaa"",
""ccc-ddd-ccc-dddd-cccc""
],
""entityColumns"": [
{
""entityId"": ""144"",
""joinColumnName"": ""device_uid"",
""columnName"": ""device_random_date""
}
],
""columnNames"": [
""ts"",
""device_uid"",
""1"",
""32"",
""55"",
""203"",
""204""
],
""startUnixTsMs"": ""1583413637000"",
""endUnixTsMs"": ""1583413640000"",
""columnFilters"": [
{
""filterType"": ""eq"",
""columnName"": ""55"",
""value"": ""1234""
}
],
""sortOrder"": [
{
""column"": ""ts"",
""order"": ""DESC""
},
{
""column"": ""55"",
""order"": ""ASC""
}
],
""entityFilters"": [
{
""entityId"": ""144"",
""entityEntryIds"": [
""12345-221-232-1231-123456""
]
}
]
}
}",
Parsed_JSON = Json.Document(body),
BuildQueryString = Uri.BuildQueryString(Parsed_JSON),
Quelle = Json.Document(Web.Contents("http://localhost:8101/device-data-reader-api/read-paginated/xxx-xxx-yyyy-yyyy", [Headers=[#"Content-Type"="application/json"], Content = Text.ToBinary(BuildQueryString)]))
in
Quelle
I tried to remove the quotes of the numbers, but this leads to the same issue, as system complains it cannot convert numbers into text.
I need the body which needs to be handed over with the request in order to do a POST request. What I'm doing wrong?
Since you seem to want to send this as application/json, I think you would change this bit in your code:
Content = Text.ToBinary(BuildQueryString)
to:
Content = Text.ToBinary(body)
and then you'd also get rid of the lines below (since you don't need them):
Parsed_JSON = Json.Document(body),
BuildQueryString = Uri.BuildQueryString(Parsed_JSON),
I don't think you would need Uri.BuildQueryString unless you wanted to send as application/x-www-form-urlencoded (i.e. URL encoded key-value pairs).
Unrelated: If it helps, you can build the structure in M and then use JSON.FromValue to turn the structure into bytes which can be put directly into the POST body. Untested example is below.
let
body = [
page = "1",
pageSize = "100",
requestParams = [
deviceUids = {
"xxx-yyy-xxx-yyyy-xxxx",
"yyy-xxx-yyy-xxxx-yyyy",
"aaa-bbb-aaa-bbbb-aaaa",
"ccc-ddd-ccc-dddd-cccc"
},
entityColumns = {
[
entityId = "144",
joinColumnName = "device_uid",
columnName = "device_random_date"
]
},
columnNames = {
"ts",
"device_uid",
"1",
"32",
"55",
"203",
"204"
},
startUnixTsMs = "1583413637000",
endUnixTsMs = "1583413640000",
columnFilters = {
[
filterType = "eq",
columnName = "55",
value = "1234"
]
},
sortOrder = {
[
column = "ts",
order = "DESC"
],
[
column = "55",
order = "ASC"
]
},
entityFilters = {
[
entityId = "144",
entityEntryIds = {
"12345-221-232-1231-123456"
}
]
}
]
],
Quelle = Json.Document(
Web.Contents(
"http://localhost:8101/device-data-reader-api/read-paginated/xxx-xxx-yyyy-yyyy",
[
Headers = [#"Content-Type" = "application/json"],
Content = Json.FromValue(body)
]
)
)
in
Quelle
It might look a little weird (since M uses [] instead of {}, {} instead of [] and = instead of :), but just mentioning in case it helps.

How to search in anonymous and nested array using find or findAll in groovy's closures using REST-Assured library?

I have following JSON response anonymous body and I need to parse nested arrays dynamically to retrieve a key's value based on a condition by using find or findAll in the groovy's closures
[
{
"children": [
{
"attr": {
"reportId": "1",
"reportShortName": "ABC",
"description": "test,
}
},
{
"attr": {
"reportId": "2",
"reportShortName": "XYZ",
"description": "test",
}
}
}
]
I've tried the following ways and had no luck to retrieve the reportId key's value from the JSON response
package com.src.test.api;
import static io.restassured.RestAssured.given;
import io.restassured.path.json.JsonPath;
import io.restassured.response.Response;
public class GetReportId {
public void getReportId(String reportName) throws Exception {
String searchReports = "http://localhost:8080/reports";
Response resp=given().request().when().get(searchReports).then().extract().response();
JsonPath jsonPath = new JsonPath(resp.asString());
String reportId1 =jsonPath.get("$.find{it.children.contains(restAssuredJsonRootObject.$.children.find{it.attr.reportShortName == 'ABC'})}.attr.reportId");
String reportId2 = jsonPath.get("$.find{it.children.attr.reportShortName.contains(restAssuredJsonRootObject.$.children.find{it.attr.reportShortName.equals('XYZ')}.attr.reportShortName)}.attr.reportId");
System.out.println("ReportId: " + reportId1);
}
}
There could be multiple JSON objects in the parent anonymous array and need to make use of find or findAll within the groovy closures to get the reportId
Need to get the reportId, but seems that something is wrong. Any help would be appreciated.
Assuming you want all the reportIds
List<String> reportIds = jsonPath.get("children.flatten().attr.reportId");
will give you what you want, even it the parent anonymous array has multiple entries.
I tested with the following JSON
[
{
"children": [
{
"attr": {
"reportId": "1",
"reportShortName": "ABC",
"description": "test"
}
},
{
"attr": {
"reportId": "2",
"reportShortName": "XYZ",
"description": "test"
}
}
]
},
{
"children": [
{
"attr": {
"reportId": "3",
"reportShortName": "DEF",
"description": "test"
}
},
{
"attr": {
"reportId": "4",
"reportShortName": "IJK",
"description": "test"
}
}
]
}
]
and it gives me ["1", "2", "3", "4"] i.e. reportIds from all the children
If you know the index of the reportId you're looking for then you can use it like so:
String reportId = jsonPath.get("children.flatten().attr.reportId[0]");
If you're looking for the reportId of a particular report you can do that too:
String reportId = jsonPath.get("children.flatten().attr.find{it.reportShortName == 'ABC'}.reportId")
will give you "1".
Note: The type of the variable you assign the results to are important for type inference and casting. For example, you CANNOT do:
String [] reportIds = jsonPath.get("children.flatten().attr.reportId");
or
int reportId = jsonPath.get("children.flatten().attr.reportId[0]");
Both those things will throw a ClassCastException.

Resources