I have one topic DemoTopic and having 2 subscriptions 'sub1' and sub2
my message payload is like this
{
"data": [
{
"id": "1",
"name": "a",
"pid": "p1"
},
{
"id": "2",
"name": "b",
"pid": "p2"
},
{
"id": "3",
"name": "c",
"pid": "p3"
}
]
}
if pid value is p1 and p2 then send messsage to sub1
how to create a filter for this using p1 and p2 value?
Subscriptions can only filter messages on some system properties and user/custom properties (aka headers). If your message contains the data required for filtering, you should promote those property values to the headers when dispatching messages.
Related
I want to query Azure CosmosDb documents with SQL API query. These Documents shall be filtered and grouped by specific values. From these groups only the document with a specified max value shall be returned.
Example
Azure CosmosDb Documents
{
"id": "1",
"someValue": "shall be included",
"group": "foo",
"timestamp": "1668907312"
}
{
"id": "2",
"someValue": "shall be included",
"group": "foo",
"timestamp": "1668907314"
}
{
"id": "3",
"someValue": "shall be included",
"group": "bar",
"timestamp": "1668907312"
}
{
"id": "4",
"someValue": "don't include",
"group": "bar",
"timestamp": "1668907312"
}
Query
I want do get all documents
with "someValue": "shall be included"
grouped by group
from group only max of timestamp
Example response
{
"id": "2",
"someValue": "shall be included",
"group": "foo",
"timestamp": "1668907314"
},
{
"id": "3",
"someValue": "shall be included",
"group": "bar",
"timestamp": "1668907312"
}
Question
What is the best way to do this? It would be optimal if
it is possible in one query
and executable with Azure SDK with use of SqlParameter (to prevent injection)
What i've tried
My current approach consists of 2 queries and uses ARRAY_CONTAINS, which does not allow the use of SqlParameter for the document paths.
{
"id": "2",
"some-value": "shall be included",
"group": "foo",
"timestamp": "1668907314"
}
First Query
SELECT c.group AS group
MAX(c.timestamp) AS maxValue
FROM c
WHERE c.someValue = 'shall be included'
GROUP BY c.group
Second Query
SELECT * FROM c WHERE ARRAY_CONTAINS(
<RESULT-FIRST-QUERY>,
{
"group": c.group,
"maxValue": c.timestamp
},
false
)
I would utilize the MAX() function in conjunction with GROUP BY i.e
SELECT *
FROM c
WHERE c.someValue = "shall be included"
GROUP BY c.group
HAVING MAX(c.timestamp)
Haven't run that yet/need to make a collection, but seems like it should do the trick...
If until activity skipped after 5 mins. I need to send a mail.
How to send the message from until to email notification pipeline.
Email notification have message as a parameter. What is the correct Syntax if pipeline skipped
<p>Hi All,<\/p>\r\n<p>Below pipeline got failed please find the error details.<\/p><br \/>\r\n<p>Pipeline Name : #{pipeline().Pipeline}<\/p>\r\n<p>Error Detail : <br\/>#{activity('LKP_INF_JOB_STATUS').output.error.message}<\/p>\r\n<p> <\/p>\r\n<p>Thanks,<br\/>SPC Support Team<\/p>\r\n<p> <\/p>\r\n<p><br \/>Note:This is an auto-generated email from XYZ, please do not reply directly to this email.<\/p>
[![enter image description here][1]][1]
[1]: https://i.stack.imgur.com/yUtUQ.png
I implemented a simple solution for this:
1. Give your Until Activity a specific timeout
2. Create a Web Activity that uses the ADF API to query the Until Activity using a Completion dependency (blue arrow)
Refer to this Stack Overflow for details on how to use this API in ADF:
How to get output parameter from Executed Pipeline in ADF?
URL: https://management.azure.com/subscriptions/#{pipeline().parameters.SubscriptionId}/resourceGroups/#{pipeline().parameters.ResourceGroupName}/providers/Microsoft.DataFactory/factories/#{pipeline().DataFactory}/pipelineruns/#{pipeline().RunId}/queryActivityruns?api-version=2018-06-01
Body:
{
"lastUpdatedAfter": "2018-06-16T00:36:44.3345758Z",
"lastUpdatedBefore": "#{utcnow()}",
"filters": [
{
"operand": "ActivityName",
"operator": "Equals",
"values": [
"Until Timeout after 1 min"
]
}
]
}
3. Use output from activity in Switch to determine which Email Activity to use (Failure, or TimeOut)
Expression:
#activity('Check Until Activity Status').output.value[0].status
4. Email Activity
{
"personalizations": [
{
"to": [
{
"email": "YourEmail#blah.com",
"name": "blah"
}
],
"cc": [
{
"email": "jane_doe#example.com",
"name": "Jane Doe"
}
],
"bcc": [
{
"email": "james_doe#example.com",
"name": "Jim Doe"
}
]
}
],
"from": {
"email": "blah#example.com",
"name": "Blah"
},
"subject": "SkipEmail",
"content": [
{
"type": "text/html",
"value": "This is a Skip Example, put whatever here"
}
]
}
I have two types of docs. Main docs and additional info for it.
{
"id": "371"
"name": "Mike",
"location": "Paris"
},
{
"id": "371-1",
"age": 20,
"lastname": "Piterson"
}
I need to merge them by id, to get result doc. The result should look like:
{
"id": "371"
"name": "Mike",
"location": "Paris"
"age": 20,
"lastname": "Piterson"
}
Using COLLECT / INTO, SPLIT(), and MERGE():
FOR doc IN collection
COLLECT id = SPLIT(doc.id, '-')[0] INTO groups
RETURN MERGE(MERGE(groups[*].doc), {id})
Result:
[
{
"id": "371",
"location": "Paris",
"name": "Mike",
"lastname": "Piterson",
"age": 20
}
]
This will:
Split each id attribute at any - and return the first part
Group the results into sepearate arrays (groups)
Merge #1: Merge all objects into one
Merge #2: Merge the id into the result
See REMOVE & INSERT or REPLACE for write operations.
I am using the karate framework for writing some automated test cases. I'd like to validate the schema for each element in a nested array list . For the example below, I would like to validate each child of each element in the returned array. Is there a way to get an array list of all children of all elements? I can do that by calling some java functions, but I was wondering if there's a way in karate to get that.
Something like "for each element in the returned array validate the schema of each of its children".
Thanks!
[
{
"id": "A",
"children": [
{
"size": "10",
"type": "A",
"name": "B"
},
{
"size": "10",
"type": "A",
"name": "B"
}
]
},
{
"id": "B",
"children": [
{
"size": "10",
"type": "A",
"name": "B"
},
}
"size": "3",
"type": "C",
"name": "D"
}
]
}
]
match each will be more convenient for validating JSON array with a schema,
* def children = $response[*].children[*]
* def schema = { "name": "#string","size": "#string","type": "#string"}
* match each children == schema
This will extract all the values of the children and validate each child is matching with the schema
Say I just wanted to get a list of all transactions that involved a specific asset (assuming I would need a query for this?). Should I use the TransactionRegistry or the Historian? What's the difference?
we have a current issue open for Historian to show history of changes/ deltas for a particular asset - https://github.com/hyperledger/composer/issues/991 As a workaround you can do the following - so for sample network trade-network with an asset Commodity (and a transaction class 'Trade') you could create a query eg:
query selectTransaction {description: "choose specific commodity asset"
statement: SELECT org.acme.biznet.Trade
WHERE (commodity == _$commodity ) }
On the difference:
Historian records all transaction activities (eg. create Asset, create participant, create identity etc etc - and also business network specific custom transactions like 'TransferAsset' or 'PlaceOrder') including (where assets / participants are concerned) what changed.
For the TransactionRegistry itself (ie a particular class - say 'TransferAsset' or 'PlaceOrder') this is stored in the Transaction registry for that class - you may have many Transaction classes in your business network. But nothing in here would be related to other activities, such as system activities, that also recorded in Historian records.
to query - you would do something like this (in a query file for example):
query myTransactions{
description: "return all transactions made (ie system transactions)"
statement: SELECT org.acme.sample.PlaceOrder
}
ie SELECT org.acme.sample.NAME_OF_TRANSACTION_CLASS
For Historian queries - eg
SELECT org.hyperledger.composer.system.HistorianRecord WHERE (transactionType == 'myTranType'
see more examples here -> https://www.ibm.com/developerworks/cloud/library/cl-create-powerful-blockchain-queries-with-hyperledger-composer/index.html
furthermore, to see the transaction data (deltas) for the asset id you're zoning in on - ie available through the transactionInvoked field of the transaction class (eg org.acme.trading.Trade transaction class). you could use REST APIs with loopback filters -eg either (both return a promise below):
return this.httpClient.get('http://localhost:3000/api/Trade?filter=%7B%22include%22%3A%22resolve%22%7D', {withCredentials: true}).toPromise();`
or
return this.httpClient.get('http://localhost:3000/api/Trade?filter=%7B%22include%22%3A%22resolve%22%7D').toPromise();
which has the {"include":"resolve"} filter to resolve relationships in the transaction class - each resolved transaction has the transaction deltas. Then you could look for the asset id in question.
Sample unresolved transaction class (below, followed by resolved txn class):
Not resolved:
[
{
"$class": "org.acme.mynetwork.Trade",
"commodity": "resource:org.acme.mynetwork.Commodity#1",
"newOwner": "resource:org.acme.mynetwork.Trader#2",
"transactionId": "354dca97fc6ac00aabbd923883e3ec2a3d09b8c75a54a8f536a88b6df31e8a0f",
"timestamp": "2018-03-23T12:02:11.228Z"
},
{
"$class": "org.acme.mynetwork.Trade",
"commodity": "resource:org.acme.mynetwork.Commodity#2",
"newOwner": "resource:org.acme.mynetwork.Trader#1",
"transactionId": "9da43acca718633ac8870e6ea34c3c9f481194e48bcdba42673570177091809f",
"timestamp": "2018-03-23T12:02:31.294Z"
}
]
Resolved with {"include":"resolve"} as a filter:
[
{
"$class": "org.acme.mynetwork.Trade",
"commodity": {
"$class": "org.acme.mynetwork.Commodity",
"tradingSymbol": "1",
"description": "werwer",
"mainExchange": "wrrewer",
"quantity": 10,
"owner": {
"$class": "org.acme.mynetwork.Trader",
"tradeId": "2",
"firstName": "tes2t",
"lastName": "test"
}
},
"newOwner": {
"$class": "org.acme.mynetwork.Trader",
"tradeId": "2",
"firstName": "tes2t",
"lastName": "test"
},
"transactionId": "354dca97fc6ac00aabbd923883e3ec2a3d09b8c75a54a8f536a88b6df31e8a0f",
"timestamp": "2018-03-23T12:02:11.228Z"
},
{
"$class": "org.acme.mynetwork.Trade",
"commodity": {
"$class": "org.acme.mynetwork.Commodity",
"tradingSymbol": "2",
"description": "Ut fugiat.",
"mainExchange": "ACE2",
"quantity": 10,
"owner": {
"$class": "org.acme.mynetwork.Trader",
"tradeId": "1",
"firstName": "test",
"lastName": "test"
}
},
"newOwner": {
"$class": "org.acme.mynetwork.Trader",
"tradeId": "1",
"firstName": "test",
"lastName": "test"
},
"transactionId": "9da43acca718633ac8870e6ea34c3c9f481194e48bcdba42673570177091809f",
"timestamp": "2018-03-23T12:02:31.294Z"
}
]