Still learning ES. This is my query:
{
"query": {
"wildcard" : {
"identifier" : "*NZ*"
}
}
}
And I have definitely document with a field named identifier and a string containing NZ. they are returned when I drop the query expression.
But I always receive 0 hits when querying this. Why?
Related
On a daily basis, I'm pushing data (time_series) to Elasticsearch. I created an index pattern, and my index have the name: myindex_* , where * is today date (an index pattern has been setup). Thus after a week, I have: myindex_2022-06-20, myindex_2022-06-21... myindex_2022-06-27.
Let's assume my index is indexing products' prices. Thus inside each myindex_*, I have got:
myindex_2022-06-26 is including many products prices like this:
{
"reference_code": "123456789",
"price": 10.00
},
...
myindex_2022-06-27:
{
"reference_code": "123456789",
"price": 12.00
},
I'm using this query to get the reference code and the corresponding prices. And it works great.
const data = await elasticClient.search({
index: myindex_2022-06-27,
body: {
query: {
match: {
"reference_code": "123456789"
}
}
}
});
But, I would like to have a query that if in the index of the date 2022-06-27, there is no data, then it checks, in the previous index 2022-06-26, and so on (until e.g. 10x).
Not sure, but it seems it's doing this when I replace myindex_2022-06-27 by myindex_* (not sure it's the default behaviour).
The issue is that when I'm using this way, I got prices from other index but it seems to use the oldest one. I would like to get the newest one instead, thus the opposite way.
How should I proceed?
If you query with index wildcard, it should return a list of documents, where every document will include some meta fields as _index and _id.
You can sort by _index, to make elastic search return the latest document at position [0] in your list.
const data = await elasticClient.search({
index: myindex_2022-*,
body: {
query: {
match: {
"reference_code": "123456789"
}
}
sort : { "_index" : "desc" },
}
});
I am new to MongoDB and I need to apply a regex to the find method using NodeJS Mongoose. The db contains elements such as:
{
"_id" : ObjectId("1234567899788675432454677"),
"nodes" : "[{\"name\":\"BINS\",\"oxygenation\":7.8,\"oxygenSaturation\":86,\"temperature\":13.3},{\"name\":\"CEST\",\"oxygenation\":4.6,\"oxygenSaturation\":52,\"temperature\":14.7},
"orp" : "0",
"chlorophyll" : "0",
"date" : "1615995776000",
"__v" : 0 }
and I need to go inside the nodes string to get all "name" values (in the example BINS and CEST). The regex that allows me to do it is
/(?<=name\\\"\:\\")\w+(?=\\)/gm
but I get errors when trying to use it both in mongo and Mongoose. For mongo I tried to escape all special characters:
db.my_collection.find( { nodes: { $regex: "\(\?<=\\\"name\\\": \\\"\)\\w\+\(\?=\\\\\)" } } )
but I get an empty result.
On Mongoose I tried several escaping such as the following, but I always get an error of invalid regex.
db.my_collection.find({ nodes : { $regex : '\(?<=name\\\\\\\"\\:\\\\\"\)\\w+\(?=\\\\\)' } }).
exec(function (err, waterSensorsReadings) { ... });
Let say I have a collection of Person{email: 'actual email', ..other data} and want to query if Person exists with given email and retrieve it data if so or get a null if not.
If i want to do that once than no problem just do a query, through mongoose using Person.findOne() or whatever.
But what if I have to do a check for 25-100 given emails? Of course I can just send a tons of requests to mongodb and retrieve the data but it seems a vast of network.
Is there a good and perfomant way to query a mongodb with multiple clauses in single batch like findBatch([{email: 'email1'}, {email: 'email2'}...{email: 'emailN'} ]) and got as result [document1,null,document3,null, documentN] where null is for not matched find criterias?
Currently I see only one option:
Huge find with single {email: $in: [] } query and that do a matching through the searching on the server side in application logic. Cons: quite cumbersome and error prone if you have more than one search criteria.
Is there any better ways to implement such thing?
Try this:
Replace the arrayOfEmails with your query array
Replace emailField with the actual name in your db documents
db.collName.aggregate([
{
"$match" : {
"emailField" : {
"$in" : arrayOfEmails
}
}
},
{
"$group" : {
"_id" : null,
"docs" : {
"$push" : {
"$cond" : [
{
"$in" : [
arrayOfEmails,
[
"$emailField"
]
]
},
"$$ROOT",
null
]
}
}
}
}
])
I have a requirement to generate a unique number (ARN) in this format
DD/MM/YYYY/1, DD/MM/YYYY/2
and insert these in elastic search index.
The approach i am thinking of is to create an auto increment field in the doc and use it to generate a new entry and use the new auto generated number to create the ARN and update the doc.
doc structure that i am planning to use:
{ id: 1, arn: 17/03/2018/01 }
something like this.
How can i get auto increment field in elastic search?
It can't be done in a single step. First you have to insert the record into the database, and then update the ARN with it's id
There is no auto-increment equivalent, for example, to hibernate id generator. You could use the Bulk API (if you have to save multiple documents at a time) and increase the _id and the ending of your ARN value programmatically.
Note: if you want to treat your id as a number, you should implement it yourself (in this example, I added a new field "my_id", because the _id of the documents is treated as a string.
POST /bulk
{ "index" : { "_index" : "your_index", "_type" : "your_type", "_id" : "1" } }
{ "arn" : "2018/03/17/1", my_id: 1 }
{ "index" : { "_index" : "your_index", "_type" : "your_type", "_id" : "2" } }
{ "arn" : "2018/03/17/2", my_id: 2 }
Then, the next time that you want to save new documents, you query for the maximum id something like:
POST /my_index/my_type/_search?size=1
{
"query": {
"fields": ["my_id"],
"sort": [{
"my_id": { "order": "desc" } }
]
}
}
If your only requirement is that this ARN should be unique, you could also let elasticsearch calculate your _id by simply not setting it. Then you could relay at some unique token generator (UID.randomUUID().toString() if work with java). Pseudo code follows:
String uuid = generateUUID() // depends on the programming language
String payload = "{ \"arn\" : + uuid + "}" // concatenate the payload
String url = "http://localhost:9200/my_index" // your target index
executePost(url, payload) // implement the call with some http client library
consider the following records:
user record
{
"_id" : ObjectId("5234ccb7687ea597eabee677"),
"class" : [
{ "_id" : ObjectId("5234ccb7687ea597eabee671", "num" : 10, "color" : "blue" },
{ "_id" : ObjectId("5234ccb7687ea597eabee672", "num" : 100, "color" : "blue" }
]
}
this user has two class sub records, now I need a query that finds all users that have class property where "class._id" has a value of at least one users "class._id"
here is a more detail example:
suppose there is four user:
A:{_id:432645624232345,class:[{_id:123,name:'foo'}]}
B:{_id:432645624232555,class:[{_id:555,name:'foo'},{_id:123,name:'foo'}]}
C:{_id:432645344232345,class:[{_id:555,name:'foo'},{_id:111,name:'www'}]}
D:{_id:432644444232345,class:[{_id:222,name:'sss'},{_id:555,name:'www'},{_id:123,name:'foo'}]}
now if B login , I need to query all the user whose class subdocument contains at least one document which's _id==555 or _id==123 (555 and 123 come from B user), in this case the query result should be:
A:{_id:432645624232345,class:[{_id:123,name:'foo'}]} // match _id=123
B:{_id:432645624232555,class:[{_id:555,name:'foo'},{_id:123,name:'foo'}]} //match _id=123 and _id=555
C:{_id:432645344232345,class:[{_id:555,name:'foo'},{_id:111,name:'www'}]} //match _id=555
D:{_id:432644444232345,class:[{_id:222,name:'sss'},{_id:555,name:'www'},{_id:123,name:'foo'}]} ///match _id=123 and _id=555
which is all the user.
so far i get this:
{"class._id" : { $in : ["5234ccb7687ea597eabee671", "5234ccb7687ea597eabee672"] } }
but when different user login the class._id query condition is different. So is there any operator to do this
{"class._id" : { $in : req.user.class } }
hope I made myself clear.
In order to achieve what you want, first you must isolate the class _ids in an array, and then use it in the query argument.
var classIds = [];
var i = 0;
while (i < req.user.class.length) {
classIds.push(req.user.class[i]._id);
i++;
}
After that you can use classIds array in the query:
{"class._id" : { $in : classIds } }
The following query condition would give you all the users that have at least one class with id equal to any of the elements in the given array:
{"class._id" : { $in : ["5234ccb7687ea597eabee671", "5234ccb7687ea597eabee672"] } }
In the array for the $in clause you may provide any id's you needed , comma separated.
In addition, if you needed such, the below query condition should check for existence of nested document within "class" property that has a property "_id" :
{ "class._id" : { $exists : true } }
Both conditions should work no matter if "class._id" is a single-valued property or an array (mongo supports that).