My code :
search_parm ={
"query": {
"match" : { "message" : "*error*" }
}
res = es.search(index='indice_1', body=search_parm)
result = res['hits']['hits']
print(result)
I am trying to get the messages which has the text 'error' in it.
The search parameter fetches the below result:
'message': 'Error handling attributes invalid syntax (<unknown>, line 1)'
But, the below value is not fetched :
'message': 'Sent message on queue: documentprocessing_error'
Mapping of message field :
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
Please let me know how I have to edit the search parameter. Thanks.
Modified my search parameter to
search_parm1 = {"query": {"wildcard": {"message": { "value": "*error*","boost": 1.0,"rewrite": "constant_score"}}}}
Working as expected. Thanks.
Related
I am having a data field in my index in which,
I want only doc 2 as result i.e logically where b comes before
a in the array field data.
doc 1:
data = ['a','b','t','k','p']
doc 2:
data = ['p','b','i','o','a']
Currently, I am trying terms must on [a,b] then checking the order in another code snippet.
Please suggest any better way around.
My understanding is that the only way to do that would be to make use of Span Queries, however it won't be applicable on an array of values.
You would need to concatenate the values into a single text field with whitespace as delimiter, reingest the documents and make use of Span Near query on that field:
Please find the below mapping, sample document, the query and response:
Mapping:
PUT my_test_index
{
"mappings": {
"properties": {
"data":{
"type": "text"
}
}
}
}
Sample Documents:
POST my_test_index/_doc/1
{
"data": "a b"
}
POST my_test_index/_doc/2
{
"data": "b a"
}
Span Query:
POST my_test_index/_search
{
"query": {
"span_near" : {
"clauses" : [
{ "span_term" : { "data" : "a" } },
{ "span_term" : { "data" : "b" } }
],
"slop" : 0, <--- This means only `a b` would return but `a c b` won't.
"in_order" : true <--- This means a should come first and the b
}
}
}
Note that slop controls the maximum number of intervening unmatched positions permitted.
Response:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.36464313,
"hits" : [
{
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.36464313,
"_source" : {
"data" : "a b"
}
}
]
}
}
Let me know if this helps!
I am new to MongoDB and NodeJS,
When i try to create the JsonSchema with data types, string, integer, date and bool, it is created but always throwing an error as document validation error while inserting the data, So i changed the bsonType of one data type to number, then it started creating collection records, but the observation is it is storing as Double datatype, I read somewhere in the stackoverflow, that it stores like that only, but my question is why is this behavior? WHY THE ERROR IS NOT BEING THROWN AT THE TIME OF CREATION OF THE JSONSCHEMA but it is throwing at the time of data insertion?
Also, if we have nested objects let us say, Customer object with Address as nested object, the main object's int/number values are stored as Double where as inside the address object's pincode storing as Int32. This is also very confusing. what is the difference between these objects but the structure of the schema is same.
What are the other ways to implement and having proper validated schema for MongoDB.
>
db.getCollectionInfos({name:"companysInt1s1"})
[
{
"name" : "companysInt1s1",
"type" : "collection",
"options" : {
"validator" : {
"$jsonSchema" : {
"bsonType" : "object",
"required" : [
"tin"
],
"properties" : {
"tin" : {
"bsonType" : "int",
"minLength" : 2,
"maxLength" : 11,
"description" : "must be a string and is not required, should be 11 characters length"
}
}
}
}
},
"info" : {
"readOnly" : false,
"uuid" : UUID("27cba650-7bd3-4930-8d3e-7e6cbbf517db")
},
"idIndex" : {
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "invoice.companysInt1s1"
}
}
]
> db.companysInt1s1.insertOne({tin:22222})
2019-02-14T15:04:28.712+0530 E QUERY [js] WriteError: Document failed validation :
WriteError({
"index" : 0,
"code" : 121,
"errmsg" : "Document failed validation",
"op" : {
"_id" : ObjectId("5c653624e382c2ec16c16893"),
"tin" : 22222
}
})
WriteError#src/mongo/shell/bulk_api.js:461:48
Bulk/mergeBatchResults#src/mongo/shell/bulk_api.js:841:49
Bulk/executeBatch#src/mongo/shell/bulk_api.js:906:13
Bulk/this.execute#src/mongo/shell/bulk_api.js:1150:21
DBCollection.prototype.insertOne#src/mongo/shell/crud_api.js:252:9
#(shell):1:1
Am i missing something or any other documentation should i be following? Appreciate your guidance...
You need to insert as NumberInt.
when you run this
db.companysInt1s1.insertOne({tin:22222})
you are actually inserting tin as float.
so the correct way to do it is
db.companysInt1s1.insertOne({tin: NumberInt(22222) })
Hi I am new to groovy and API Automation. I have the following Json and i want to add assertion to check cyclestartdate and cycleEnddate based on sequence number.
{
"status" : "success",
"locale" : "",
"data" : {
"periods" : [
{
"payCycleId" : "custompayperiod",
"sequence" : 1,
"cycleStartDate" : "2018-10-01",
"cycleEndDate" : "2018-10-08"
},
{
"payCycleId" : "custompayperiod",
"sequence" : 2,
"cycleStartDate" : "2018-10-09",
"cycleEndDate" : "2018-10-16"
}
]
}
}
How do i check if sequence 1 cycleStartDate is 2018-10-01
Groovy provides JsonSlurper class that makes parsing JSON documents easier. Consider following example that reads JSON document as a String (it supports different initialization methods as well):
import groovy.json.JsonSlurper
def inputJson = '''{
"status" : "success",
"locale" : "",
"data" : {
"periods" : [
{
"payCycleId" : "custompayperiod",
"sequence" : 1,
"cycleStartDate" : "2018-10-01",
"cycleEndDate" : "2018-10-08"
},
{
"payCycleId" : "custompayperiod",
"sequence" : 2,
"cycleStartDate" : "2018-10-09",
"cycleEndDate" : "2018-10-16"
}
]
}
}'''
def json = new JsonSlurper().parseText(inputJson)
assert json.data.periods.find { it.sequence == 1 }.cycleStartDate == '2018-10-01'
Having JSON document loaded you can extract data by accessing nested fields. For instance, json.data.periods gives you an access to the array stored in your JSON document. Then method find { it.sequence == 1 } returns a node from this array where sequence field is equal to 1. And lastly you can extract cycleStartDate and compare it with the expected date.
You can find more useful examples in Groovy's official documentation.
{
TypeList" : [
{
"TypeName" : "Carrier"
},
{
"TypeName" : "Not a Channel Member"
},
{
"TypeName" : "Service Provider"
}
]
}
Question :
db.supplies.find("text", {search:"\"chann\" \"mem\""})
For above query I want display :
{
TypeName" : "Not a Channel Member"
}
But I am unable to get my result.
What are changes I have to do in query .
Please help me.
The below query will return your desired result.
db.supplies.aggregate([
{$unwind:"$TypeList"},
{$match:{"TypeList.TypeName":{$regex:/.*chann.*mem.*/,$options:"i"}}},
{$project:{_id:0, TypeName:"$TypeList.TypeName"}}
])
If you can accept to get an output like this:
{
"TypeList" : [
{
"TypeName" : "Not a Channel Member"
}
]
}
then you can get around using the aggregation framework which generally helps performance by running the following query:
db.supplies.find(
{
"TypeList.TypeName": /chann.*mem/i
},
{ // project the list in the following way
"_id": 0, // do not include the "_id" field in the output
"TypeList": { // only include the items from the TypeList array...
$elemMatch: { //... where
"TypeName": /chann.*mem/i // the "TypeName" field matches the regular expression
}
}
})
Also see this link: Retrieve only the queried element in an object array in MongoDB collection
This question already has answers here:
MongoDB Query Help - query on values of any key in a sub-object
(3 answers)
Closed 6 years ago.
This is my data set, which is part of a bigger json code. I want to write a query, which will match all fields inside the value chain.
Dataset:
"value_chain" : {
"category" : "Source, Make & Deliver",
"hpe_level0" : "gift Chain Planning",
"hpe_level1" : "nodemand to Plan",
"hpe_level2" : "nodemand Planning",
"hpe_level3" : "nodemand Sensing"
},
Example:
If someone searches for "gift", the query should scan through all fields, and if there is a match, return the document.
This is something I tried, but didnt work
db.sw_api.find({
value_chain: { $elemMatch: { "Source, Make & Deliver" } }
})
Sounds like you need to create $text index on all the text fields first since it performs a text search on the content of the fields indexed with a text index:
db.sw_api.createIndex({
"value_chain.category" : "text",
"value_chain.hpe_level0" : "text",
"value_chain.hpe_level1" : "text",
"value_chain.hpe_level2" : "text",
"value_chain.hpe_level3" : "text"
}, { "name": "value_chain_text_idx"});
The index you create is a composite index consisting of 5 columns, and mongo will automatically create the text namespace for you by default if you don't override it. With the above, if you don't specify the index name as
db.sw_api.createIndex({
"value_chain.category" : "text",
"value_chain.hpe_level0" : "text",
"value_chain.hpe_level1" : "text",
"value_chain.hpe_level2" : "text",
"value_chain.hpe_level3" : "text"
});
there is a potential error "ns name is too long (127 byte max)" since the text index will look like this:
"you_db_name.sw_api.$value_chain.category_text_value_chain.hpe_level0_text_value_chain.hpe_level1_text_value_chain.hpe_level2_text_value_chain.hpe_level3_text"
Hence the need to give it a name which is not too long if autogenerated by mongo.
Once the index is created, a db.sw_api.getIndexes() query will show you the indexes present:
/* 1 */
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "dbname.sw_api"
},
{
"v" : 1,
"key" : {
"_fts" : "text",
"_ftsx" : 1
},
"name" : "value_chain_text_idx",
"ns" : "dbname.sw_api",
"weights" : {
"value_chain.category" : 1,
"value_chain.hpe_level0" : 1,
"value_chain.hpe_level1" : 1,
"value_chain.hpe_level2" : 1,
"value_chain.hpe_level3" : 1
},
"default_language" : "english",
"language_override" : "language",
"textIndexVersion" : 3
}
]
Once you create the index, you can then do a $text search:
db.sw_api.find({ "$text": { "$search": "gift" } })