Currently, we have have a patch endpoint for jsonschema and now we want to limit the operation that we can do while applying the patch.
The patch endpoint looks like this:
def patch(self, name, version):
...
schema = Schema.get(name, version)
serialized_schema = schema.patch_serialize()
...
data = request.get_json()
patched_schema = apply_patch(serialized_schema, data)
serialized_data, errors = update_schema_serializer.load(
patched_schema, partial=True)
...
data example:
[
{ "op": "replace", "path": "/config/notifications/", "value": "foo" },
{ "op": "add", "path": "/config/", "value": {} },
{ "op": "remove", "path": "/config/notifications/"}
]
Here we want to only have add, remove and replace operation. Is there any way to only apply patch for these operation and not for copy, move and test?
In your schema definition, define op as an enum with only the options that you want
Related
I am in no way an expert with groovy so please don't hold that against me.
I have JSON that looks like this:
{
"metrics": [
{
"name": "metric_a",
"help": "This tracks your A stuff.",
"type": "GAUGE",
"labels": [
"pool"
],
"unit": "",
"aggregates": [],
"meta": [
{
"category": "CAT A",
"deployment": "environment-a"
}
],
"additional_notes": "Some stuff (potentially)"
},
...
]
...
}
I'm using it as a source for automated documentation of all the metrics. So, I'm iterating through it in various ways to get the information I need. So far so good, I'm most of the way there. The problem is this all needs to be organized per the deployment environment. Meaning, multiple metrics will share the same value for deployment.
My thought was I could create a map with deployment as the key and the metric name for any metric that has a matching deployment as the value. Once I have that map, it should be easy for me to organize things the way they should be. I can't figure out how to do that. The result is all the metric names are added which is expected since I'm not doing anything to filter them out. I was thinking that groupBy would make sense here but I can't figure out how to use it effectively and frankly I'm not sure it will solve my problem by itself. Here is my code so far:
parentChild = [:]
children = []
metrics.each { metric ->
def metricName = metric.name
def depName = metric.meta.findResult{ it.deployment }
children.add(metricName)
parentChild.put(depName, children)
}
What is the best way to create a new map where the values for each key are based off a specific condition?
EDIT: The desired result would be each key in the resulting map would be a unique deployment value from all the metrics (as a string). Each value would be name of each metric that contains that deployment (as an array).
[environment-a:
[metric_a,metric_b,metric_c,...],
environment-b:
[metric_d,metric_e,metric_f,...]
...]
I would use a combo of withDefault() to pre-fill each map-entry value with a fresh TreeSet-instance (sorted no-duplicates set) and standard inject().
I reduced your sample data to the bare minimum and added some new nodes:
import groovy.json.*
String input = '''\
{
"metrics": [
{
"name": "metric_a",
"meta": [
{
"deployment": "environment-a"
}
]
},
{
"name": "metric_b",
"meta": [
{
"deployment": "environment-a"
}
]
},
{
"name": "metric_c",
"meta": [
{
"deployment": "environment-a"
},
{
"deployment": "environment-b"
}
]
},
{
"name": "metric_d",
"meta": [
{
"deployment": "environment-b"
}
]
}
]
}'''
def json = new JsonSlurper().parseText input
def groupedByDeployment = json.metrics.inject( [:].withDefault{ new TreeSet() } ){ res, metric ->
metric.meta.each{ res[ it.deployment ] << metric.name }
res
}
assert groupedByDeployment.toString() == '[environment-a:[metric_a, metric_b, metric_c], environment-b:[metric_c, metric_d]]'
If your metrics.meta array is supposed to have a single value, you can simplify the code by replacing the line:
metric.meta.each{ res[ it.deployment ] << metric.name }
with
res[ metric.meta.first().deployment ] << metric.name
I use an API that returns a response with the following structure:
{
"data": [
{
"id": "id-1",
"type": "objectType",
"attributes": { ... },
"links":
"linksDataAndMetadate": {...}
}, ...
],
"links":
"thisLink": "some url",
"thatLink": "another url"
}
We use jsonapi-serializer package in our code base for serialization/deseiralization, but when I use it to deserialize the response, the outer links part (the ones that includes "thisLink" and "thatLink") overwrites the "links" field inside the object attributes. Is there a way to use the package so that it will have a different behavior?
I have the following JSON response for my REST endpoint:
{
"response": {
"status": 200,
"startRow": 0,
"endRow": 1,
"totalRows": 1,
"next": "",
"data": {
"id": "workflow-1",
"name": "SampleWorkflow",
"tasks": [
{
"id": "task-0",
"name": "AWX",
"triggered_by": ["task-5"]
},
{
"id": "task-1",
"name": "BrainStorming",
"triggered_by": ["task-2", "task-5"]
},
{
"id": "task-2",
"name": "OnHold",
"triggered_by": ["task-0", "task-4", "task-7", "task-8", "task9"]
},
{
"id": "task-3",
"name": "InvestigateSuggestions",
"triggered_by": ["task-6"]
},
{
"id": "task-4",
"name": "Mistral",
"triggered_by": ["task-3"]
},
{
"id": "task-5",
"name": "Ansible",
"triggered_by": ["task-3"]
},
{
"id": "task-6",
"name": "Integration",
"triggered_by": []
},
{
"id": "task-7",
"name": "Tower",
"triggered_by": ["task-5"]
},
{
"id": "task-8",
"name": "Camunda",
"triggered_by": ["task-3"]
},
{
"id": "task-9",
"name": "HungOnMistral",
"triggered_by": ["task-0", "task-7"]
},
{
"id": "task-10",
"name": "MistralIsChosen",
"triggered_by": ["task-1"]
}
]
}
}
}
I am using rest-assured with a groovy gpath expression for an extraction as follows:
given()
.when()
.get("http://localhost:8080/workflow-1")
.then()
.extract()
.path("response.data.tasks.findAll{ it.triggered_by.contains('task-3') }.name")
which correctly gives me [Mistral, Ansible, Camunda]
What I am trying to achieve is to find the task names that are triggered by the InvestigateSuggestions task. But I don't know for sure that the taskId I have to pass in to contains() is task-3; I only know its name i.e. InvestigateSuggestions. So I attempt to do:
given()
.when()
.get("http://localhost:8080/workflow-1")
.then()
.extract()
.path("response.data.tasks.findAll{
it.triggered_by.contains(response.data.tasks.find{
it.name.equals('InvestigateSuggestions')}.id) }.name")
which does not work and complains that the parameter "response" was used but not defined.
How do I iterate over the outer collection from inside the findAll closure to find the correct id to pass into contains() ??
You can make use of a dirty secret, the restAssuredJsonRootObject. This is undocumented (and subject to change although it has never change as far as I can remember in the 7 year+ lifespan of REST Assured).
This would allow you to write:
given()
.when()
.get("http://localhost:8080/workflow-1")
.then()
.extract()
.path("response.data.tasks.findAll{
it.triggered_by.contains(restAssuredJsonRootObject.response.data.tasks.find{
it.name.equals('InvestigateSuggestions')}.id) }.name")
If you don't want to use this "hack" then you need to do something similar to what Michael Easter proposed in his answer.
When it comes to generating matchers based on the response body the story is better. See docs here.
I'm not sure if this is idiomatic but one approach is to find the id first and then substitute into another query:
#Test
void testCase1() {
def json = given()
.when()
.get("http://localhost:5151/egg_minimal/stacko.json")
// e.g. id = 'task-3' for name 'InvestigateSuggestions'
def id = json
.then()
.extract()
.path("response.data.tasks.find { it.name == 'InvestigateSuggestions' }.id")
// e.g. tasks have name 'task-3'
def tasks = json
.then()
.extract()
.path("response.data.tasks.findAll{ it.triggered_by.contains('${id}') }.name")
assertEquals(['Mistral', 'Ansible', 'Camunda'], tasks)
}
I have a For_Each loop in an Azure Logic App that calls another, nested, Logic App. The result from each iteration of the nested Logic Apps is a JSON object that contains an array of strings, like this:
{
"Results": ["string a", "string b"]
}
So the output from my For_Each loop in the parent Logic App looks like this:
[
{"Results": ["string a", "string b"]},
{"Results": ["string c", "string d"]}
]
I want to put all these strings into a single flat list that I can pass to another action.
How can I do this? Is it possible using the workflow definition language and built-in functions, or do I need to use an external function (in a service, or an Azure Function)?
There's a simpler solution, working with Array Variables.
At the top level, outside the For Each loop, declare a variable with an InitializeVariable action:
"Initialize_Items_variable": {
"inputs": {
"variables": [
{
"name": "Items",
"type": "Array",
"value": []
}
]
},
"runAfter": {},
"type": "InitializeVariable"
}
Inside the For Each, use a AppendToArrayVariable action. You can append the Response object of the Nested Logic App you just called.
"Append_to_Items_variable": {
"inputs": {
"name": "Items",
"value": "#body('Nested_Logic_App_Response')"
},
"runAfter": {
},
"type": "AppendToArrayVariable"
}
Hope it helps.
Picking up on #DerekLi's useful comment above, it seems this is not possible at the time of writing with Logic Apps schema version 2016-06-01.
One of the great strengths of Logic Apps is the ability to leverage the power of Azure Functions to solve problems like this that can't (yet) be solved in the schema language.
Re-writing the array is trivial in c# within a function:
using System.Net;
public class Result
{
public List<string> Results {get; set;}
}
public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
log.Info("C# HTTP trigger function processed a request.");
var inputs = await req.Content.ReadAsAsync<List<Result>>();
var outputs = new List<string>();
foreach(var item in inputs)
{
log.Info(item.Results.ToString());
outputs.AddRange(item.Results.Where(x => !string.IsNullOrEmpty(x)));
}
return req.CreateResponse(HttpStatusCode.OK, outputs);
}
And this function can then be passed the result of the For_Each loop:
"MyFunction": {
"inputs": {
"body": "#body('Parse_JSON')",
"function": {
"id": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Web/sites/{function-app-name}/functions/{function-name}"
},
"method": "POST"
},
"runAfter": {
"For_each": [
"Succeeded"
]
},
"type": "Function"
}
There is also a way to do it using the workflow definition language. (https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-workflow-definition-language).
Using the fonctions string and replace you can work on your json as a string rather than on objects.
Here is a Flat_List action that follows a Parse_JSON action with your data:
Your data:
[
{"Results": ["string a", "string b"]},
{"Results": ["string c", "string d"]}
]
Flat_List component:
"Flat_List": {
"inputs": "#replace(replace(replace(string(body('Parse_JSON')),']},{\"Results\":[',','),'}]','}'),'[{','{')",
"runAfter": {
"Parse_JSON": [
"Succeeded"
]
},
"type": "Compose"
},
What happens here? First we use string that takes your json data and gives:
[{"Results":["string a", "string b"]},{"Results":["string c", "string d"]}]
We replace all the ]},{"Results":[ by ,.
We replace all the }] by }.
We replace all the [{ by {.
We get the string {"Results":["string a","string b","string c","string d"]}
Then you are free to parse it back to json with:
"Parse_JSON_2": {
"inputs": {
"content": "#outputs('Flat_List')",
"schema": {
"properties": {
"Results": {
"items": {
"type": "string"
},
"type": "array"
}
},
"type": "object"
}
},
"runAfter": {
"Flat_List": [
"Succeeded"
]
},
"type": "ParseJson"
}
You can see it as a proof of concept as the Azure Function may be easier to re-read later but there may be many reason not to want to instantiate a new Azure Function while you can do the job in Logic App.
Feel free to ask for more details if needed :)
This technique works pretty well, and only uses run-of-the-mill Logic App actions:
1. start with declaring an empty array variable (action Variable: Initialise variable)
2. iterate through your items (action Control: For each), e.g. the resultset from a previous action
in each iteration, first compose the JSON fragment you need (action Data Operations: Compose)
then append the output of your Compose action to the array (action: Variable: Append to array variable)
3. then, outside the loop, join the elements of the array (action Data Operations: Join)
4. do what you need with the output of the Join action, e.g. send as response payload (action Request: Response)
This is what it looks like in the end:
You can use #body(nestedLogicApp) outside of the for-each loop to access all the nested Logic Apps' response in an array.
I have the web-form builder for science events. The event moderator creates registration form with arbitrary amount of boolean, integer, enum and text fields.
Created form is used for:
register a new member to event;
search through registered members.
What is the best search tool for second task (to search memebers of event)? Is ElasticSearch well for this task?
I wrote a post about how to index arbitrary data into Elasticsearch and then to search it by specific fields and values. All this, without blowing up your index mapping.
The post is here: http://smnh.me/indexing-and-searching-arbitrary-json-data-using-elasticsearch/
In short, you will need to do the following steps to get what you want:
Create a special index described in the post.
Flatten the data you want to index using the flattenData function:
https://gist.github.com/smnh/30f96028511e1440b7b02ea559858af4.
Create a document with the original and flattened data and index it into Elasticsearch:
{
"data": { ... },
"flatData": [ ... ]
}
Optional: use Elasticsearch aggregations to find which fields and types have been indexed.
Execute queries on the flatData object to find what you need.
Example
Basing on your original question, let's assume that the first event moderator created a form with following fields to register members for the science event:
name string
age long
sex long - 0 for male, 1 for female
In addition to this data, the related event probably has some sort of id, let's call it eventId. So the final document could look like this:
{
"eventId": "2T73ZT1R463DJNWE36IA8FEN",
"name": "Bob",
"age": 22,
"sex": 0
}
Now, before we index this document, we will flatten it using the flattenData function:
flattenData(document);
This will produce the following array:
[
{
"key": "eventId",
"type": "string",
"key_type": "eventId.string",
"value_string": "2T73ZT1R463DJNWE36IA8FEN"
},
{
"key": "name",
"type": "string",
"key_type": "name.string",
"value_string": "Bob"
},
{
"key": "age",
"type": "long",
"key_type": "age.long",
"value_long": 22
},
{
"key": "sex",
"type": "long",
"key_type": "sex.long",
"value_long": 0
}
]
Then we will wrap this data in a document as I've showed before and index it.
Then, the second event moderator, creates another form having a new field, field with same name and type, and also a field with same name but with different type:
name string
city string
sex string - "male" or "female"
This event moderator decided that instead of having 0 and 1 for male and female, his form will allow choosing between two strings - "male" and "female".
Let's try to flatten the data submitted by this form:
flattenData({
"eventId": "F1BU9GGK5IX3ZWOLGCE3I5ML",
"name": "Alice",
"city": "New York",
"sex": "female"
});
This will produce the following data:
[
{
"key": "eventId",
"type": "string",
"key_type": "eventId.string",
"value_string": "F1BU9GGK5IX3ZWOLGCE3I5ML"
},
{
"key": "name",
"type": "string",
"key_type": "name.string",
"value_string": "Alice"
},
{
"key": "city",
"type": "string",
"key_type": "city.string",
"value_string": "New York"
},
{
"key": "sex",
"type": "string",
"key_type": "sex.string",
"value_string": "female"
}
]
Then, after wrapping the flattened data in a document and indexing it into Elasticsearch we can execute complicated queries.
For example, to find members named "Bob" registered for the event with ID 2T73ZT1R463DJNWE36IA8FEN we can execute the following query:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "flatData",
"query": {
"bool": {
"must": [
{"term": {"flatData.key": "eventId"}},
{"match": {"flatData.value_string.keyword": "2T73ZT1R463DJNWE36IA8FEN"}}
]
}
}
}
},
{
"nested": {
"path": "flatData",
"query": {
"bool": {
"must": [
{"term": {"flatData.key": "name"}},
{"match": {"flatData.value_string": "bob"}}
]
}
}
}
}
]
}
}
}
ElasticSearch automatically detects the field content in order to index it correctly, even if the mapping hasn't been defined previously. So, yes : ElasticSearch suits well these cases.
However, you may want to fine tune this behavior, or maybe the default mapping applied by ElasticSearch doesn't correspond to what you need : in this case, take a look at the default mapping or, for even further control, the dynamic templates feature.
If you let your end users decide the keys you store things in, you'll have an ever-growing mapping and cluster state, which is problematic.
This case and a suggested solution is covered in this article on common problems with Elasticsearch.
Essentially, you want to have everything that can possibly be user-defined as a value. Using nested documents, you can have a key-field and differently mapped value fields to achieve pretty much the same.