How do I convert this query to query String in url?
The goal is to apply the filter passed as a query in Postman request and convert it to url, so the query is in the url and not separate of it.
{
"query": {
"bool": {
"filter": [
{
"range": {
"timestamp": {
"gte": "2019-08-12T06:00:00",
"lt": "2019-08-12T07:00:00"
}
}
}
]
}
}
}
You can serialise your JSON into a URL-encoded string, and then pass that encoded string as one big query parameter. This parameter could then be decoded back to JSON in your backend.
Using something like this might help.
Related
I am using ElasticSearch 7.9 with a nodejs client. I have the following query :
{
"query": {
"bool": {
"must":[
{ "terms" : { "id" : ["5f0d06fb5112231eb89eb819", "5f0d06fb5112231eb89eb817"] } },
{"query_string": {
"query": "(News) OR (Entertainent)",
"fields": [ "topics", "subTopics", "categories"]
}
},
{
"multi_match": {
"query": "publisher",
"fields": ["text", "name", "title", "subtitle", "description"]
}
}
]
}
}
}
I want to be able to conditionally add the terms filter for id if list of ids coming in to the nodejs/js function is not empty. Similarly for the multi_match query text as well. Add the multi_filter only if the incoming text is not empty
Should all queries be pre-constructed or is it possible to have conditional blocks and add only if the empty text or array of Ids are not empty.
My current method expects both ids and text input to the method to have valid values but these could be empty. Do I separate methods to handle the empty conditions
export const searchResults = async (text, ids) => {
const response = await client.search({
index: "new_index", //customer.id
type: "_doc",
body: {
query: {
bool: {
must: [
{terms: {"id": ids}},
{query_string: {
query: "(News) OR (Entertainent)",
fields: [ "topics", "subTopics", "categories"]
}
},
{
multi_match: {
query: text,
fields: ["text", "name", "title", "subtitle", "description"],
}
}
],
}
}
},
});
return response?.hits?.hits.map(({_source}) => _source) || [];
};
any help is really appreciated.
In a production environment and managing client based applications you should use search-templates, where you can use conditional blocks of queries. Besides, if you would want to change your query you would not neet to redeploy your app, just change it on ES.
I have a JAVA code that fetches some records from ES. The JAVA code is as follows :
public Map<String, Object> getRiskDetailsFromEventIds(final String eventId) {
try {
String elasticBaseUrl = elasticsearchConfig.getEsUrl();
String esIndexName = elasticsearchConfig.getHitsIndex();
String endpointUrl = elasticBaseUrl + "/" + esIndexName + "/_search";
String queryTemplate = IOUtils.toString(ESRepositary.class.getClassLoader().getResourceAsStream(
"querytemplates/hits_search_event_es_query_template.json"), Charset.defaultCharset());
String query = String.format(queryTemplate, eventId);
return getResultsFromElastic(endpointUrl, query);
} catch (IOException ex) {
throw new UncheckedIOException(ex);
}
}
My hits_search_event_es_query_template.json looks something like this
{
"_source": [
"hitNumber","eventId","hitScore","severityLevel","priority","hitType.typeCode","targetOwner","hitType.search","recommendedAction","referralId"
],
"query": {
"bool": {
"must": [
{
"terms": {
"eventId": ["%s"]
}
}
]
}
},
"size" : "50"
}
Now, this takes a String. My eventId is : "A,B,C,D"
The String that I need to pass to the query template should be : "A","B","C","D"
Can someone help me break down my String this way or if someone can help me modify my ES query to utilise what I have.
Getting a List of the eventIds, I am able to do the following :
String eventId = eventIds.stream().map(event -> "\""+event+"\"").collect(Collectors.joining(","));
where eventIds is the List
Also I have changed my query as below :
{
"_source": [
"hitNumber","eventId","hitScore","severityLevel","priority","hitType.typeCode","targetOwner","hitType.search","recommendedAction","referralId"
],
"query": {
"bool": {
"must": [
{
"terms": {
"eventId": [%s]
}
}
]
}
},
"size" : "50"
}
This gives me the required query which I need.
I'm trying to create a JSON schema that can support validating JSON objects with property values that can either be regular JSON types OR strings representing valid JSONpath expressions.
So for example, given this schema:
{
"$schema": "http://json-schema.org/draft-07/schema",
"properties": {
"age": {
"type": "number"
}
}
}
Either of these JSON objects could be valid:
{
"age": 30
}
{
"age" "$.age"
}
I've gotten stuck trying to add a custom keyword called jsonPath like this:
{
"$schema": "http://json-schema.org/draft-07/schema",
"properties": {
"age": {
"type": "number",
"jsonPath": true
}
}
}
ajv.addKeyword('jsonPath', {
valid: true,
compile: () => data => {
return /^\$./.test(data)
}
})
Ideally I would love to just be able to check if a given property value is a valid JSONPath string and if so, approve it. Otherwise let ajv run it's own validation.
Thanks for any help!
I don't know if you can prevent other keywords from running. There are multiple ways to apply checks in JSON Schema to the same location, so this would likely be pretty difficult and probably not something that's supported by ajv.
You could build this into your schema.
{
"$schema": "http://json-schema.org/draft-07/schema",
"properties": {
"age": {
"anyOf": [
{
"type": "number"
},
{
"pattern": "REGEX FOR JSON PATH"
}
]
}
}
}
You could de-duplicate the regex by using definitions and referencing it using $ref.
I have documents like the below in my elasticsearch
{"method":"POST","url":"/saas/services/1.0/account/*/purchase/*/subscription/cancel"}
{"method":"POST","url":"/saas/services/1.0/account/*/purchase/*/cancel"}
I am searching these documents using node.js client. Below is the query I am sending.
Example 1 type : DSL
client.search({
index: 'my_node',
body: {
query: {
bool: {
must: {
match: { method: 'POST' },
match: { url: '/saas/services/1.0/account/*/purchase/*/cancel' }
}
}
}
}
}
Output
I am getting the other record with subscription cancel. The query is not matching the exact document. I tried the below query , it worked for this case but not working for few other test cases.
document
{"method":"POST","url":"/paas/service/1.0/act/*/purchase"}}
{"method":"GET","url":"/paas/service/1.0/act/*/purchase"}}
{"method":"PUT","url":"/paas/service/1.0/act/*/purchase"}}
Example 2 type : query string
client.search({
index: 'my_node',
q: 'method: POST AND url: /saas/services/1.0/account/*/purchase'
}
Output
I get the GET purchase document regardless of other two. Tried for few other documents, the method argument is not getting recognized.
How do I write elasticsearch query to search through the documents to match both the properties of method and url. i had tried query time boosting for the url but doesn't seem to work.
Edit - Mapping information
{
"nodeclient": {
"mappings": {
"logs": {
"properties": {
"method": {
"type": "string" },
"url": {
"type": "string" }
} } }
}
}
Try add quotation marks
client.search({ index: 'my_node', body: { "query": { "bool": { "must": {
"match": { "method": 'POST' }, "match": { "url":
'/saas/services/1.0/account//purchase//cancel' } } } } } }
I tried optimizing the query with Mirage plugin for elasticsearch queries. I was trying with different examples and saw the responses. Below is my final query that works for my scenario
{
"query": {
"bool": {
"must": [
{ "match_phrase": { "url": uri }},
{ "match": { "method": http } }
]}
}
match_phrase suited better for the url parameter. Also I missed the array part inside the must clause.
I have an endpoint that I am proxying into ElasticSearch API for a simple user search I am conducting.
/users?nickname=myUsername&email=myemail#gmail.com&name=John+Smith
Somet details about these parameters are the following
All parameters are optional
nickname can be searched as a full text search (i.e. 'myUser' would return 'myUsername')
email must be an exact match
name can be searched as full text search for each token (i.e. 'john' would return 'John Smith')
The ElasticSearch search call should treat the parameters collectively as AND'd.
Right now, I am not truly sure where to start as I am able to execute the query on each of the parameters alone, but not all together.
client.search({
index: 'users',
type: 'user',
body: {
"query": {
//NEED TO FILL THIS IN
}
}
}).then(function(resp){
//Do something with search results
});
First you need to create the mapping for this particular use case.
curl -X PUT "http://$hostname:9200/myindex/mytype/_mapping" -d '{
"mytype": {
"properties": {
"email": {
"type": "string",
"index": "not_analyzed"
},
"nickname": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}'
Here by making email as not_analyzed , you are making sure only the exact match works.
Once that is done , you need to make the query.
As we have multiple conditions , it would be a good idea to use bool query.
You can combine multiple queries and how to handle them using bool query
Query -
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "qbox"
}
},
{
"prefix": {
"nickname": "qbo"
}
},
{
"match": {
"email": "me#qbox.io"
}
}
]
}
}
}
Using the prefix query , you are telling Elasticsearch that even if the token starts with qbo , qualify it as a match.
Also prefix query might not be very fast , in that case you can go for ngram analyzer - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-ngram-tokenizer.html