I would like to create a metric that represents a list of groups. The group contains a name and the count of elements.
public class GroupMetric{
String name;
int count;
}
This metric is intended to be provided via a Rest API. The GroupMetric objects are already created and the data are set.
Questions:
Is it possible to display such a collection list of groups with one metric or is a metric required for each group?
How can I query the metric from the MeterRegistry?
Is it possible to use pagination, because the GroupMetric objects can be greater than 100.000
The output should look like this:
{
"items": [
{
"name": "group1",
"count": "10000"
},
{
"name": "group2",
"count": "54"
},
{
"name": "group3",
"count": "122"
}
],
"totalItems": 3,
"currentPage": 0,
"totalPages": 1,
"itemCount": 3
}
Currently using spring boot and micrometer.
Does anyone have experience or a good idea how to implement this?
Related
I have implemented the Azure Maps search at https://learn.microsoft.com/en-gb/rest/api/maps/search/getsearchaddress but I want to get a list of only certain "types".
In the results below, the type is "Street", but I am interested in returning only those where the type matches "MunicipalitySubdivision".
If I do a call to this service, the API returns results in blocks of 10 by default (which can be upped to 200), and gives a TotalResults field as well. It is possible to iterate through (for example) 50,000 results 200 at a time by providing a results offset startIndex parameter in the API, but this doesn't seem like the most efficient way to return just results of one type.
Can anyone suggest anything?
{
"type": "Street",
"id": "GB/STR/p0/1199538",
"score": 5.07232,
"address": {
"streetName": "Hampton Road",
"municipalitySubdivision": "Birmingham, Aston",
"municipality": "Birmingham",
"countrySecondarySubdivision": "West Midlands",
"countrySubdivision": "ENG",
"postalCode": "B6",
"extendedPostalCode": "B6 6AB,B6 6AE,B6 6AN,B6 6AS",
"countryCode": "GB",
"country": "United Kingdom",
"countryCodeISO3": "GBR",
"freeformAddress": "Hampton Road, Birmingham",
"countrySubdivisionName": "England"
},
"position": {
"lat": 52.50665,
"lon": -1.90082
},
"viewport": {
"topLeftPoint": {
"lat": 52.50508,
"lon": -1.90015
},
"btmRightPoint": {
"lat": 52.50804,
"lon": -1.90139
}
}
}
There currently isn't an option to limit the results as you requested other than to scan the results programmatically. If the address information you have is structured (you have the individual pieces) and is not a freeform string, then using the structured geocoding service would allow you to specify the type right request when passing in the address parts: https://learn.microsoft.com/en-us/rest/api/maps/search/getsearchaddressstructured
I have 3 models. 2 resource models, account(id, name) and widget(id, name), and 1 mapping model to map between the two widget_to_account(id, account_id, widget_id), to tell what widgets an account has access to, so to speak.
When stating the relationship between the models in their JSONs, using the guide in http://loopback.io/doc/en/lb3/HasManyThrough-relations.html, RESTful requests like "get widgets of account id=1" for example, works perfectly.
GET /accounts/1/widgets yields the widgets that account 1 has, yielding a widgets array:
[
{
"id": 1,
"name": "wg_user_mgr"
},
{
"id": 2,
"name": "wg_desc"
}
]
That's all good.
However, say I wanted to append this widgets array result along with the account object returned by a GET to the account model?. Loopback documentation suggests that this is done using the include keyword with the request, like so:
GET /accounts/1?filter[include]=widgets, returning hopefully an account model with it's allowed widgets:
{
"id": 1,
"name": "Account1Name",
"widgets": [
{
"id": 1,
"name": "wg_user_mgr",
"display_name": "User Manager"
},
{
"id": 2,
"name": "wg_desc",
"display_name": "Description"
}
]
}
However, what is actually returned by loopback with that request, is:
{
"id": 1,
"name": "Account1Name",
"widgets": []
}
Empty widgets array! When I look at the loopback SQL debugs, I see that it does go to the widget_to_account table and selects the entries of account_id=1, but interestingly it stop there and just returns an empty widgets array.
Any clues? The hasManyThrough loopback docs doesn't actually show any examples of using include like this to bridge two models that are connected via a mapping model.
My guess is they just forgot to code it in ¯\_(ツ)_/¯
UPDATE:
Doing some more digging around, I found the answer at https://groups.google.com/forum/#!topic/loopbackjs/sH7bKoqzU5c.
Where you define the relationships in the 2 resource models, you have to specifically define the "keyThrough" value.
NOT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account"
}
}
BUT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account",
"keyThrough": "account_id"
}
}
This is not made super clear, and is even stated incorrectly in the loopback api docs -.-
UPDATE:
Doing some more digging around, I found the answer at https://groups.google.com/forum/#!topic/loopbackjs/sH7bKoqzU5c.
Where you define the relationships in the 2 resource models, you have to specifically define the "keyThrough" value.
NOT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account"
}
}
BUT THIS:
"relations": {
"widgets": {
"type": "hasMany",
"model": "widget",
"foreignKey": "account_id",
"through": "widget_to_account",
"keyThrough": "widget_id"
}
}
This is not made super clear, and is even stated incorrectly in the loopback api docs. I wish they'de stop this "auto-naming" paradigm they've been pushing around. Looking at loopback SO and the wider community, it's generally caused so much pain with models being named incorrectly, keys like this being set to totally arbitary names -.-
I have a document stored in ElasticSearch as below.
_source:
{
"firstname": "John",
"lastname": "Smith",
"medals":[
{
"bucket": 100,
"count": 1
},
{
"bucket": 150,
"count": 2
}
]
}
I can access the string type value inside a document using doc.firstname for scripted metric aggregation http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html.
But I am not able to get the field value using doc.medals[0].bucket.
Can you please help me out and let me know how to access the values inside nested fields?
Use _source for nested properties.
Doc holds fields that are loaded in memory. Nested documents may not be loaded and should be accessed with _source.
For instance:
GET index/type
{
"aggs": {
"NAME": {
"scripted_metric": {
"init_script": "_agg['collection']=[]",
"map_script": "_agg['tr'].add(_source.propertry1.prop);",
"combine_script": "return _agg",
"reduce_script": "return _aggs"
}
}
},
"size": 0
}
I have the web-form builder for science events. The event moderator creates registration form with arbitrary amount of boolean, integer, enum and text fields.
Created form is used for:
register a new member to event;
search through registered members.
What is the best search tool for second task (to search memebers of event)? Is ElasticSearch well for this task?
I wrote a post about how to index arbitrary data into Elasticsearch and then to search it by specific fields and values. All this, without blowing up your index mapping.
The post is here: http://smnh.me/indexing-and-searching-arbitrary-json-data-using-elasticsearch/
In short, you will need to do the following steps to get what you want:
Create a special index described in the post.
Flatten the data you want to index using the flattenData function:
https://gist.github.com/smnh/30f96028511e1440b7b02ea559858af4.
Create a document with the original and flattened data and index it into Elasticsearch:
{
"data": { ... },
"flatData": [ ... ]
}
Optional: use Elasticsearch aggregations to find which fields and types have been indexed.
Execute queries on the flatData object to find what you need.
Example
Basing on your original question, let's assume that the first event moderator created a form with following fields to register members for the science event:
name string
age long
sex long - 0 for male, 1 for female
In addition to this data, the related event probably has some sort of id, let's call it eventId. So the final document could look like this:
{
"eventId": "2T73ZT1R463DJNWE36IA8FEN",
"name": "Bob",
"age": 22,
"sex": 0
}
Now, before we index this document, we will flatten it using the flattenData function:
flattenData(document);
This will produce the following array:
[
{
"key": "eventId",
"type": "string",
"key_type": "eventId.string",
"value_string": "2T73ZT1R463DJNWE36IA8FEN"
},
{
"key": "name",
"type": "string",
"key_type": "name.string",
"value_string": "Bob"
},
{
"key": "age",
"type": "long",
"key_type": "age.long",
"value_long": 22
},
{
"key": "sex",
"type": "long",
"key_type": "sex.long",
"value_long": 0
}
]
Then we will wrap this data in a document as I've showed before and index it.
Then, the second event moderator, creates another form having a new field, field with same name and type, and also a field with same name but with different type:
name string
city string
sex string - "male" or "female"
This event moderator decided that instead of having 0 and 1 for male and female, his form will allow choosing between two strings - "male" and "female".
Let's try to flatten the data submitted by this form:
flattenData({
"eventId": "F1BU9GGK5IX3ZWOLGCE3I5ML",
"name": "Alice",
"city": "New York",
"sex": "female"
});
This will produce the following data:
[
{
"key": "eventId",
"type": "string",
"key_type": "eventId.string",
"value_string": "F1BU9GGK5IX3ZWOLGCE3I5ML"
},
{
"key": "name",
"type": "string",
"key_type": "name.string",
"value_string": "Alice"
},
{
"key": "city",
"type": "string",
"key_type": "city.string",
"value_string": "New York"
},
{
"key": "sex",
"type": "string",
"key_type": "sex.string",
"value_string": "female"
}
]
Then, after wrapping the flattened data in a document and indexing it into Elasticsearch we can execute complicated queries.
For example, to find members named "Bob" registered for the event with ID 2T73ZT1R463DJNWE36IA8FEN we can execute the following query:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "flatData",
"query": {
"bool": {
"must": [
{"term": {"flatData.key": "eventId"}},
{"match": {"flatData.value_string.keyword": "2T73ZT1R463DJNWE36IA8FEN"}}
]
}
}
}
},
{
"nested": {
"path": "flatData",
"query": {
"bool": {
"must": [
{"term": {"flatData.key": "name"}},
{"match": {"flatData.value_string": "bob"}}
]
}
}
}
}
]
}
}
}
ElasticSearch automatically detects the field content in order to index it correctly, even if the mapping hasn't been defined previously. So, yes : ElasticSearch suits well these cases.
However, you may want to fine tune this behavior, or maybe the default mapping applied by ElasticSearch doesn't correspond to what you need : in this case, take a look at the default mapping or, for even further control, the dynamic templates feature.
If you let your end users decide the keys you store things in, you'll have an ever-growing mapping and cluster state, which is problematic.
This case and a suggested solution is covered in this article on common problems with Elasticsearch.
Essentially, you want to have everything that can possibly be user-defined as a value. Using nested documents, you can have a key-field and differently mapped value fields to achieve pretty much the same.
I want to simulate a parent child relation in elastic search and perform some analytics work over it. My use case is something like this
I have a shop owner like this
"_source": {
"shopId": 5,
"distributorId": 4,
"stateId": 1,
"partnerId": 2,
}
and now have child records (for each day) like this:
"_source": {
"shopId": 5,
"date" : 2013-11-13,
"transactions": 150,
"amount": 1980,
}
The parent is a record per store, while the child is the transactions each store does for
day. Now I want to do some complex query like
Find out total transaction for each day for the last 30 days where distributor is 5
POST /newdb/shopsDaily/_search
{
"query": {
"match_all": {}
},
"filter": {
"has_parent": {
"type": "shop",
"query": {
"match": {
"distributorId": "5"
}
}
}
},
"facets": {
"date": {
"histogram": {
"key_field": "date",
"value_field": "transactions",
"interval": 100
}
}
}
}
But the result I get do not take the filtering into account which I applied.
So I changed the query to this:
POST /newdb/shopDaily/_search
{
"query": {"filtered": {
"query": {"match_all": {}},
"filter": { "has_parent": {
"type": "shop",
"query": {"match": {
"distributorId": "13"
}}
}}
}},
"facets": {
"date": {
"histogram": {
"key_field": "date",
"value_field": "transactions",
"interval": 100
}
}
}
}
And then the final histogram facet took filtering into count.
So, when I browsed though I found out this is due to using filtered(which can only be used inside query clause and not outside like filter) rather than filter,
but it also mentioned that to have fast search you should use filter. Will searching as I did in second step (when I used filtered instead of filter) effect the performance of elastic search? If so, how can I make my facets honor filters and not effect the performance?
Thanks for you time
filters in Filtered query (filters in query clause) are cached, hence faster. These type of filters affect both search result and facet counts.
Filters outside the query clause are not considered during facet calculations. They are considered only for search results. Facet is calculated only on the query clause. If you want filtered facets then you need to set filters to each of the facet clauses.