using terms in elasticsearch js - node.js

because of the wonderful documentation Elasticsearch has , I cant figure out the proper syntax to search for a term , this is my code:
let checkuser = await client.search({
index: "users",
type: "my_users",
body: {
query: {
term: {
email: req.body.email
}
}
}
});
I wish to search for an object that has a key value pair of 'email' with a certain email, but I wish it to be the exact email I wrote, if its a#mail.com ab#mail.com should not match, I know I need to use terms but when i write it like that it doesn't work, whats wrong with my syntax?
PS this is my index mapping:
"users" : {
"mappings" : {
"jobix_users" : {
"properties" : {
"confirmed" : {
"type" : "boolean"
},
"email" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"firstName" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"jobNotification" : {
"type" : "boolean"
},
"jobTitle" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"lastName" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"password" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"userName" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}

You use default mapping and this means that you use standard tokenizer when you index documents.
How you can see in mapping email field have two interpretations:
text
keyword
In text standard tokenizer is work and save tokens to your index. This mean you can find for term alex and for term mail.com. If you want to find whole email then your query should look as:
{
"query": {
"term": {
"email.keyword": req.body.email
}
}
}
But elasticsearch has special uax_url_email tokenizer for urls and mails. I would like to recommend use this tokenizer for the email field.

Related

Mongo: filter documents from multiple collections and merge

I am new to mongo and NodeJS and have a use case where I want to get filtered results from multiple collection.
Advance apologies for the long post.
for ex:
collectionA
{
"_id" : "foo#gmail.com",
"name" : "Foo",
"location" : {
"coordinates" : [
-122.420170,
37.780080
],
"type" : "Point"
}
},
{
"_id" : "bar#gmail.com,
"name" : "Bar",
"location" : {
"coordinates" : [
-122.420060,
37.780180
],
"type" : "Point"
}
}
collectionB: Some attributes are not present for all the documents and hence optional
{
"_id" : "foo#gmail.com"
"AttributeA" : [
{
"name" : "AttA_Name",
"val" : "Coll_B_AttA_Val_Foo"
},
{
"name" : "AttA_Name1",
"val" : "Coll_B_AttA_Val_Foo"
}]
},
{
"_id" : "bar#gmail.com"
"AttributeA" : [
{
"name" : "AttA_Name",
"val" : "Coll_B_AttA_Val_Bar"
},
{
"name" : "AttA_Name2",
"val" : "Coll_B_AttA_Val_Bar"
}
],
"AttributeB" : [
{
"name" : "AttB_Name",
"val" : "Coll_B_AttB_Val_Bar"
}
]
}
CollectionC: Some attributes are not present for all the documents and hence optional
{
"_id" : "foo#gmail.com"
"AttributeA" : [
{
"name" : "Coll_C_AttA_Name",
"val" : "Coll_C_AttA_Val_Foo"
}]
},
{
"_id" : "bar#gmail.com"
"AttributeA" : [
{
"name" : "Coll_C_AttA_Name",
"val" : "Coll_C_AttA_Val_Bar"
}
],
"AttributeB" : [
{
"name" : "Coll_C_AttB_Name",
"val" : "Coll_C_AttB_Val_Bar"
}
]
}
I know Collection B and C schema looks the same but the purpose is different and they have to be different. DB design is not the question so I would appreciate if do not put all the focus on it.
Query:
Assume there is another user (Alan) with same attributes present as Bar that exist in the collection but is not living nearby the location of Bar.
The query I am trying to build on top of these is,
Find people living nearby from CollectionA
And Collection B, if AttributeA exist and have an element with name: AttA_Name
And in Collection C, if AttributeA exist and have an name: Coll_C_AttA_Name
In the above case I am expecting a result as
{
"_id" : "foo#gmail.com",
"name" : "Foo",
"location" : {
"coordinates" : [
-122.420170,
37.780080
],
"type" : "Point"
},
"collectionB_AttributeA" : [
{
"name" : "AttA_Name",
"val" : "Coll_B_AttA_Val_Foo"
},
{
"name" : "AttA_Name1",
"val" : "Coll_B_AttA_Val_Foo"
}]
,
"collectionC_AttributeA" : [
{
"name" : "Coll_C_AttA_Name",
"val" : "Coll_C_AttA_Val_Foo"
}]
},
{
"_id" : "bar#gmail.com,
"name" : "Bar",
"location" : {
"coordinates" : [
-122.420060,
37.780180
],
"type" : "Point"
},
"collectionB_AttributeA":[
{
"name" : "AttA_Name",
"val" : "Coll_B_AttA_Val_Bar"
},
{
"name" : "AttA_Name2",
"val" : "Coll_B_AttA_Val_Bar"
}
],
"collectionC_AttributeA":[
{
"name" : "Coll_C_AttA_Name",
"val" : "Coll_C_AttA_Val_Bar"
}
]
}
There is one way of doing is in parts:
query Collection A and get the nearby people
Loop through the result of 1 and find in CollectionB if they have AttributeA and an element with name AttA_Name and eliminate if they don't match.
Loop through the filtered results from 2 and find in CollectionC if they have AttributeA and and element with name Coll_C_AttA_Name and if they don't eliminate such documents.
Is there a way I can use aggregate to build this query as one? I tried reading and trying the aggregate but seems like my understanding is incomplete.
let result = await CollectionASchema.aggregate([
{
$geoNear: {
near: { type: "Point", coordinates: [ Number(long) , Number(lat) ] },
distanceField: "dist.calculated",
minDistance: 0,
maxDistance: radiusinmetres,
spherical: true
}
},
{
$lookup:
{
from: 'collectionB',
pipeline: [
{ $match : { $and: [{ AttributeA :{$exists: true}}, { [category]: { $elemMatch: { name: “AttA_Name” } } }] }},
{ $project: { AttributeA: 0 } }
],
as: "collectionB_AttributeA"
}
}
])
If you can explain if this is possible or let me know off this is the right approach that would be helpful.

Mongo pull object from array inside array

i have inside my mongoDB collection this document
{
"_id" : ObjectId("5b633025579fac22e74bf3be"),
"FLAGS" : [
{
"toSent" : [
{
"_id" : ObjectId("5b633025579fac22e74bf3c2"),
"phone" : "+84404040404"
},
{
"_id" : ObjectId("5b633025579fac22e74bf3c1"),
"phone" : "+212652253403"
},
{
"_id" : ObjectId("5b633025579fac22e74bf3c0"),
"phone" : "+212123456788"
}
],
"_id" : ObjectId("5b633025579fac22e74bf3bf"),
"action" : "group_p_a"
},
{
"toSent" : [
{
"_id" : ObjectId("5b633031579fac22e74bf3c9"),
"phone" : "+212651077199"
},
{
"_id" : ObjectId("5b633031579fac22e74bf3c8"),
"phone" : "+84404040404"
},
{
"_id" : ObjectId("5b633031579fac22e74bf3c7"),
"phone" : "+212652253403"
},
{
"_id" : ObjectId("5b633031579fac22e74bf3c6"),
"phone" : "+212123456788"
}
],
"_id" : ObjectId("5b633031579fac22e74bf3c5"),
"action" : "group_p_a"
}
],
"time" : ISODate("2018-08-02T16:24:05.747+0000"),
"action_user_phone" : "+212123456788",
"idGroup" : "e534379a-1580-4568-b5ec-6eaf981538d2",
"nomGroup" : "MOH FOR EVER",
"__v" : NumberInt(0)
}
TODO
I need to remove for example this element { "_id" : ObjectId("5b633025579fac22e74bf3c2"), "phone" : "+84404040404"}
WHAT I DID
GroupEvents.update({}, {$pull:{FLAGS:{$elemMatch:{toSent:{phone: "+84404040404"} }}}},function(err,ret){
if(err)
console.log("error"+err);
if(ret)
console.log(ret);
});
It remove all what's inside toSent event if it doesn't match.
Any help please
You need to use $ positional operator instead of $elemMatch here
GroupEvents.update(
{ "Flags.toSent.phone": "+84404040404" },
{ "$pull": { "FLAGS.$.toSent": { "phone": "+84404040404" }}},
)
If you want remove from every element of FLAGS array this you need to use $[] the all positional operator
GroupEvents.update(
{ "Flags.toSent.phone": "+84404040404" },
{ "$pull": { "FLAGS.$[].toSent": { "phone": "+84404040404" }}},
)

Need to apply two group in sequence and second group should will have effect on result of first group

I want to group my data on the base of factoryId field and then each factory there will be multiple orders want to again group on basis of orderId as each order can contain multiple items. Here I am giving the example of my data and what I need and first group by which I tried.
{
"_id" : ObjectId("5b3e270c42d8004cea382e87"),
"factoryId" : ObjectId("5aa76190cef23a1561b8056c"),
"productId" : ObjectId("5aa78c66cef23a1561b80893"),
"orderId" : ObjectId("5b3e270c42d8004cea382e86"),
"generatedOrderId" : "3985-166770-4554",
"productName" : "Lakme Lotion"
},
{
"_id" : ObjectId("5b3e270c42d8004cea382e88"),
"factoryId" : ObjectId("5b39aed32832f72062e51c23"),
"productId" : ObjectId("5b3cb96139cec8341df52c4b"),
"orderId" : ObjectId("5b3e270c42d8004cea382e86"),
"generatedOrderId" : "3985-166770-4554",
"productName" : "Coke"
},
{
"_id" : ObjectId("5b3e27b07fe0d94d62b76b2a"),
"factoryId" : ObjectId("5aa76190cef23a1561b8057c"),
"productId" : ObjectId("5ac21075ac347a5fbf355028"),
"orderId" : ObjectId("5b3e27b07fe0d94d62b76b27"),
"generatedOrderId" : "3985-755507-7484",
"productName" : "Spoon"
}
And I want result as:
{
"factoryId":ObjectId("5aa76190cef23a1561b8057c"),
"orders":[
{
"orderId":ObjectId("5b3e270c42d8004cea382e86")
"items":[
{
"productName":"Lakme Lotion"
},
{
"productName":"Coke"
}
]
}
]
}
Can anyone help me with this?. Any help is appreciated.
I tried and It worked for me. Sorry
db.getCollection("transactions").aggregate(
[
{
"$group" : {
"_id" : "$orderId",
"items" : {
"$push" : "$$ROOT"
}
}
},
{
"$project" : {
"orderId" : "$_id",
"items" : "$items",
"_id" : 0
}
},
{
"$unwind" : {
"path" : "$items",
"preserveNullAndEmptyArrays" : false
}
},
{
"$group" : {
"_id" : "$items.factoryId",
"orders" : {
"$push" : "$$ROOT"
}
}
},
{
"$project" : {
"factoryId" : "$_id",
"orders" : "$orders",
"_id" : 0
}
}
]
);

Count where value does not match array

I have following offers collection
Here readBy contains the _id of the users...
Now I want to count the number of unRead offers for the userId = "5add82620d7f5b38240c63d4"
{
"_id" : ObjectId("5aeaab5ed6a9c97d0209260a"),
"expiresIn" : ISODate("2018-05-30T18:30:00.000Z"),
"name" : "Trip ",
"readBy" : [
ObjectId("5add82620d7f5b38240c63d4"),
ObjectId("5add82620d7f5b38240c63c6")
],
"__v" : 0
}
{
"_id" : ObjectId("5aeaab7dd6a9c97d0209260b"),
"expiresIn" : ISODate("2018-05-29T18:30:00.000Z"),
"name" : "Trip",
"readBy" : [ObjectId("5add82620d7f5b38240c63d4")],
"__v" : 0
}
{
"_id" : ObjectId("5aeae233d6a9c97d02092622"),
"expiresIn" : ISODate("2018-05-25T18:30:00.000Z"),
"name" : "two way off",
"readBy" : [],
}
{
"_id" : ObjectId("5aeae49643f10d284726069c"),
"expiresIn" : ISODate("2018-05-25T18:30:00.000Z"),
"name" : "two way off",
"readBy" : [],
}
{
"_id" : ObjectId("5aeae49743f10d284726069d"),
"expiresIn" : ISODate("2018-05-25T18:30:00.000Z"),
"name" : "two way off",
"readBy" : []
}
{
"_id" : ObjectId("5aeae49743f10d284726069e"),
"expiresIn" : ISODate("2018-05-25T18:30:00.000Z"),
"name" : "two way off",
"readBy" : []
}
so for the above collection my output should be
[{
numberOfUnreadOffers: 4
}]
because four of the collection does not have 5add82620d7f5b38240c63d4 in readBy array
You basically use $setIsSubset and $cond here:
var userId= "5add82620d7f5b38240c63d4";
Model.aggregate([
{ "$group": {
"_id": null,
"numberOfUnreadOffers": {
"$sum": {
"$cond": {
"if": {
"$setIsSubset": [[mongoose.Types.ObjectId(userId)], "$readBy"]
},
"then": 0,
"else": 1
}
}
}
}}
])
Of course you also need to "cast" using mongoose.Types.ObjectId from the "string" to a valid ObjectId value.
But really you get better performance from a simple count() instead:
Model.count({ "readBy": { "$ne": userId } })
So you really should not use .aggregate() for this at all.

how to fetch the element present inside Json Object through mongodb query

I am using mongodb database, I unable to apply query to fetch all document with status CERTIFIED,I am using below document, please suggets any query like to fetch all document with "status":"CERTIFIED"
{
"_id" : ObjectId("592fb37d57693963cca325e5"),
"json" : {
"query_by_gtin_response:queryByGtinResponse" : {
"productData" : {
"gtin" : "31212121212167",
"productDataRecord" : {
"module" : [
{
"bpi:basicProductInformationModule" : {
"xmlns:bpi" : "urn:gs1:tsd:basic_product_information_module:xsd:1",
"productName" : {
"languageCode" : "en",
"value" : "test"
},
"brandNameInformation" : {
"brandName" : {
"value" : "test pub"
}
}
}
},
{
"product_tracking_information_module" : {
"variantId" : {
"value" : "31124ff0-4693-11e7-9492-01c27f5bdb71"
},
"status" : {
"value" : "CERTIFIED"
},
"name" : {
"value" : "admin#gmail.com"
},
"createdBy" : {
"value" : "552f5b90b348147e03e49b62"
},
"createdDate" : {
"value" : "2017-06-01T06:26:05.551Z"
},
"updatedBy" : {
"value" : "552f5b90b348147e03e49b62"
},
"updatedDate" : {
"value" : "2017-06-01T06:26:05.551Z"
},
"applicationId" : {
"value" : "webapp"
},
"history" : {
"createdDate" : {
"value" : "2017-06-01T06:26:05.551Z"
},
"updatedDate" : {
"value" : "2017-06-01T06:26:05.551Z"
},
"status" : {
"value" : "CERTIFIED"
},
"updatedBy" : {
"value" : "552f5b90b348147e03e49b62"
},
"createdBy" : {
"value" : "552f5b90b348147e03e49b62"
},
"applicationId" : {
"value" : "webapp"
},
"name" : {
"value" : "admin#gmail.com"
}
}
}
}
]
}
}
}
}
}
You can use a function as your $where operator:
collection.find({ $where: function() {
var productDataRecord = this.json["query_by_gtin_response:queryByGtinResponse"]["productData"]["productDataRecord"];
var status = productDataRecord["module"][1]["product_tracking_information_module"].status;
return (status.value == "CERTIFIED")
}})

Resources