I want to be able to chain together multiple filters using JMESpath but it appears you cannot filter on the output of a filter.
My working example is as follows:
// document:
{
pips: {
ancestors:[{
id: 'p01234567'
}],
episode: {
more: 'data',
goes: 'here
}
}
}
// working filter: `[pips][?ancestors[?pid=='p01234567'] && episode]`
But I would like to write my filter instead as follows, effectively to filter the output of another filter:
[pips][?ancestors[?pid=='p01234567']][?episode]
Any idea why this doesn't work?
I am building this in NodeJS using the following NPM package: https://www.npmjs.com/package/jmespath
Is there a mistake in the syntax I am using, is there a bug in the library I am using, or am I just trying to do something that is outside what JMESpath allows?
Thank you!
I found the reason why - projections are evaluated in two steps, with the left-hand-side creating a JSON array of initial values and the right-hand-side is the expression.
Solution: "Pipe expressions" which allow you to"operate on the result of a projection".
So instead of the incorrect expression from before: [pips][?ancestors[?pid=='p01234567']][?episode]
This instead should be written as: [pips][?ancestors[?pid=='p01234567']] | [?episode]
And to undo the conversion of the initial document into an array, we can convert this back to an object like this: [pips][?ancestors[?pid=='p01234567']] | [?episode] | [0]
As a side note, I observed that using parentheses () also works, but using pipes are a bit cleaner.
Related
I'm doing the following traversal:
g.V().has('Transfer','eventName','Airdrop').as('t1').
outE('sent_to').
inV().dedup().as('a2').
inE('sent_from').
outV().as('t2').
where('t1',eq('t2')).by('address').
outE('sent_to').
inV().as('a3').
select('a3','a2').
by('accountId').toList().groupBy { it.a3 }.collectEntries { [(it.key): [a2 : it.value.a2]]};
So as you can see I'm basically doing a traversal and at the end I'm using groovy with collectEntries to aggregate the results like I need them, which is aggregated by a3 in this case. The results look like this:
==>0xfe43502662ce2adf86d9d49f25a27d65c70a709d={a2=[0x99feb505a8ed9976cf19e757a9536117e6cdc5ba, 0x22019ad32ea3adabae68003bdefd099d7e5e3886]}
(This is GOOD, because the number of values in a2 is at least 2)
==>0x129e0131ea3cc16fe5252d7280bd1258f629f20f={a2=[0xf7958fad496d15cf9fd9e54c0012504f4fdb96ff]}
(This is NOT GOOD, I want to return in my list only those combinations where there are at lest 2 values for a2)
I have tried using filters and an additional where step in the traversal itself but I haven't been able to do it. I'm not sure if this is something I should skip using Groovy in my last line. Any help or orientation would be very much appreciated
I don't think you need to drop into Groovy to get the answer you want. It would be preferable to do this all in Gremlin especially since you intend to filter results which could yield some performance benefit. Gremlin has it's own group() step as well as methods for filtering the resulting Map:
g.V().has('Transfer','eventName','Airdrop').as('t1').
out('sent_to').
dedup().as('a2').
in('sent_from').as('t2').
where('t1',eq('t2')).by('address').
out('sent_to').inV().as('a3').
select('a3','a2').
by('accountId').
group().
by('a3').
by('a2').
unfold().
where(select(values).limit(local,2).count(local).is(gte(2)))
The idea is to build your Map with group() then deconstruct it to entries with unfold(). You the filter each entry with where() by selecting the values of the entry, which is a List of "a2" then counting the items locally in that List. I use limit(local,2) to avoid unnecessary iteration beyond 2 since the filter is gte(2).
The easiest way to do this is with findAll { }.
.groupBy { it.a3 }
.findAll { it.value.a2.size() > 1 }
.collectEntries { [(it.key): [a2: it.value.a2]] }
if some a2 are null, then value.a2 also evaluates to null and filters the results without the need for explicit nullchecks
This appears to be pretty basic but I am unable to find a suitable pipeline expression function to achieve this.
I have set an array variable VAR1 with the following value, which is an output from a SQL Lookup activity in an ADF pipeline:
[
{
"Code1": "1312312"
},
{
"Code1": "3524355"
}
]
Now, I need to convert this into a comma separated string so I can pass it to a SQL query in the next activity - something like:
"'1312312','3524355'"
I am unable to find an expression function to iterate over the array elements, nor convert an array to a string. The only pipeline expression functions I see are to convert string to array and not the other way around.
Am I missing something basic? How can this be achieved?
Use 'join' function present in 'collection' functions in 'Add dynamic content'. For example:
join(variables('ARRAY_VARIABLE'), ',')
I had this same issue and was not totally satisfied just using the join function because it keeps the keys from the json object. Also, using an iterator approach can work but is needlessly expensive and slow if you have a long list. Here was my approach, using join and replace:
replace(replace(join(variables('VAR1'), ','), '{"Code1":', ''), '}', ''))
This will give you exactly the output you are looking for.
I got it working using a ForEach loop activity to iterate over my array and use a Set Variable task with a concat expression function to create my comma separated string.
Wish they had an iterator function in the expression language itself, that would have made it much easier.
In case, you just have two elements in the array, then you can do something like:
#concat(variables('variable_name')[0].Code1, ',', variables('variable_name')[1].Code1)
I need to search mongodb collection for a specific pattern field. I tried using {$exists:true}; However, this gives results only if you provide exact field.
I tried using {$exists:true} for my field. But this does not give results if you give some pattern.
{
"field1":"value1",
"field2":"value2",
"field3":object
{/arjun1/pat1: 1,
/arjun2/pat2: 3,
/arjun3/pat3: 5
}
"field4":"value4",
}
From some field, I get the keys pat3 & field3. From this I would need to find out if the value /arjun3/pat3 exists in the document.
If I use {"field3./arjun3/pat3":{$exists:true}}, this would give me results. But the problem is I get only field3 and pat3 and I need to use some pattern matching like field3.*.pat3 and then use $expr or $exists; which I'm not exactly sure how to. Please help.
you could try something of this kind
db.arjun.find(
{"field3" : {
"$elemMatch" : { $and: [
{"arjun3.pat3" : {$exists:true}},
{"arjun3.pat3" : 5}
]
}}}
);
You can either go for regex (re module) for SQL like pattern matching, and compile your own custom wildcard. But if you don't want that then you can simple use the fnmatch module, it is a builtin library of python which allows wildcard matching for multiple characters (via*) or a single character (via ?).
import fnmatch
a = "hello"
print(fnmatch.fnmatch(a, "h*"))
OUTPUT:-
True
I have this data structure in puppet:
Struct[
'ssh_keys' => Hash[
String,
Struct[
'path' => String,
'content' => String,
]
]
] $myStructure
And I would like to extract all the 'path' values into an Array.
I got as far as mapping the inner Struct using
$testvariable = $myStructure['ssh_keys'].map |$items| { $items[1] }
But a bit suck here, any help would be much appreciated.
It's not clear what you're hung up on, as you are indeed most of the way to a solution that should work. For hashes, however, I do usually prefer the form of the map() function in which the lambda takes two parameters, a separate key and value. That will read more clearly in this case:
$testvariable = $myStructure['ssh_keys'].map |$unused, $ssh_key| { $ssh_key['path'] }
But you should also be able in your original code to index $items[1] as the hash (Struct) it is: $items[1]['path'].
You could also use the dig() function if you cannot abide the mixture of array and hash indexing in the above: $items.dig(1, 'path').
for convenience in grouping couchdb functions
i created a file format that groups separate things together using yaml
it basically contains entries in the form of name.ext: |
followed by a intended block of code in the language fitting to .ext
for more pleasant editing i'd like to have vim use the correct syntax highlighters for them
edit
some code examples as requested
simple:
map.coffee: |
(doc) ->
for item in doc.items:
emit [doc.category, item], null
return
reduce: _count
more complex:
map.coffee: |
(doc) ->
emit doc.category, {items: 1, val: doc.value}
return
reduce.coffee: |
(keys, values, rereduce) ->
ret = {items: 0, val: 0}
for v in values
ret.items += doc.items
ret.val += doc.val
return ret
I believe that what you want it to make use of Vim's syntax regions (:help syn-region). But regions take delimiters as parameters.
You have a well defined start but not a defined end, maybe you could work your way around by establishing some conventions here like "2 empty new lines at the end".
There are similar answers that might give you a hint (including the docs) on how to implement a solution, like: Embedded syntax highligting in Vim
Also interesting and similar approach is this Vimtip: http://vim.wikia.com/wiki/Different_syntax_highlighting_within_regions_of_a_file
You have to write your own syntax file, and define a syntax region for each of your entries. Inside that region, you can then syntax-include the corresponding language as defined by your ext. Read all the details at :help :syn-include.
If that sounds too complicated, check out my SyntaxRange plugin. It is based on the Vimtip mentioned by alfredodeza. With it, you can quickly assign a syntax to a range of lines, e.g. :11,42SyntaxInclude perl