I'm sure this is a trivial question for the most of the Mongo users however, I was unlucky in finding the right answer.
I have a collection of documents like
{
_id:"2a1fd96c-73c5-49e1-a8ca-bd03a20c0197",
timestamp:1519725979178,
storeID:"xxx",
unitID: "yyy",
status:[1, 0, 1, 0],
_rev:"1-8019f22bf26b4d6cb99ae5460b3e0c65"
}
I need to find all documents that:
storeID = "xxx" AND unitID = "yyy" AND status[2] = 1
My filter entry that works with Compass
{'status.2': 1,storeID:'xxx',unitID:'yyy'}
However when I am trying to convert it into Js code
Model.find({'status.2': 1,storeID:'xxx',unitID:'yyy'})
Nothing is returned.
After a couple hours of pulling my hair out, I nailed the problem.
The filtering query
{'status.2':1,storeID:'xxx',unitID:'yyy'} and {'status.2':{$eq:1},storeID:'xxx',unitID:'yyy'} was actually okay.
Unfortunatelly, I used .find() along with Model instead of invoking it in collection scope:
let mongoose = require('mongoose'),
Schema = mongoose.Schema,
MyShema = new Schema({/* definition */}),
Model = mongoose.model('MyShema'),
filter = {'status.2':1,storeID:'xxx',unitID:'yyy'};
// BEFORE
let cursor = Model.find(fd); //returns total=0
// AFTER
let cursor = Model.collection.find(fd); //returns total=80
cursor.count()
.then(total=>console.log('total',total))
.catch(error=>console.log(error));
Somehow bizarrely when I removed 'status.2':1 from the filtering options, both instances of the cursor were returning an identical amount of documents.
The reason for that that was so pathetic - the status was declared in Schema as String whereas it should be obviously Array!
Related
I am new to redis and created an index and am attempting to ingest Zeek Logging Data, create an index for multiple fields, and then search fields in that index. For the life of me, I cannot get any values to return when searching for the #orig_ip name or using JSON.GET to retried any id.* related fields.
UPDATE: I figured this out after more troubleshooting and am updating here to help anyone else struggling with this problem.
Here is my WRONG code for creating the index:
# Options for index creation
index_def = IndexDefinition(
index_type=IndexType.JSON,
prefix = ['uid:'],
score = 0.5,
score_field = 'doc_score'
)
# Schema definition
schema = (
TagField('$.orig_l2_addr', as_name='orig_mac'),
TagField('$.id.orig_h', as_name='orig_ip'), #Wrong field path
TagField('$.id.resp_h', as_name='resp_ip'), #Wrong field path
NumericField('$.orig_bytes', as_name='orig_bytes'),
NumericField('$.resp_bytes', as_name='resp_bytes'),
NumericField('$.ts', as_name='timestamp')
)
r.ft('py_conn_idx').create_index(schema, definition = index_def)
Here is the result I kept getting with the above WRONG schema (no results)
search_result4 = r.ft('py_conn_idx').search(Query('#orig_ip:{192\.168\.210\.27}'))
Results for "#orig_ip:{192\.168\.210\.27}":
0
UPDATE: Working schema definition:
So it turns out even though Zeek is only using the . in field names vice using it to create an object, but the . in the field names was the culprit in my query failures. I needed to access the fields for the index as follows:
# Schema definition
schema = (
TagField('$.orig_l2_addr', as_name='orig_mac'),
TagField('$.["id.orig_h"]', as_name='orig_ip'), #Fixed field reference
TagField('$.["id.resp_h"]', as_name='resp_ip'), #Fixed field reference
NumericField('$.orig_bytes', as_name='orig_bytes'),
NumericField('$.resp_bytes', as_name='resp_bytes'),
NumericField('$.ts', as_name='timestamp')
)
After recreating the index with this schema, I get results with my query:
Results for "#orig_ip:{192\.168\.210\.27}":
Document {'id': 'uid:CPvYfTI4Zb1Afp2l5',....
Thanks to this stackoverflow question for finally walking me to the cause of my troubles: How to get objects value if its name contains dots?
Putting this answer here so this question gets marked as having one. See the updated question/code above!
Hi I have two mongodb collections. The first one returns json data (array) and with the output of this, I want to return documents that match.
When I run Console.log (req.bidder.myBids) I get the following output:
[{"productId":"3798b537-9c7b-4395-9e41-fd0ba39aa984","price":3010},{"productId":"3798b537-9c7b-4395-9e41-fd0ba39aa984","price":3020},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1040},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1050},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1060},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1070},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1090},{"productId":"4c4bd71c-6664-4d56-b5d3-6428fe1bed19","price":1100}]
The productId has duplicates, I want to remove duplicates and then call a routine that finds all the products that match and output as json.
So far I have this code that only outputs one document, but cant figure out how to add the array of productId's and then fetch all corresponding products.
var agencyId = req.body.agencyId;
var productId = req.body.productId;
if (!validate.STRING(agencyId)) {
res.apiError(messages.server.invalid_request);
} else {
dbProduct.find({productId:{$in:['3798b537-9c7b-4395-9e41-fd0ba39aa984','4c4bd71c-6664-4d56-b5d3-6428fe1bed19']}
}).then(dbRes => {
console.log(dbRes);
Updated code and works with hard-wired productId and updated above code. Looking at how to get the array data and transpose replacing the hard-wired productId's
The $in operator is what you want. See the docs here: https://docs.mongodb.com/manual/reference/operator/query/in/
Suppouse a firestore Database filled with this structure, being Events and Users collections:
Firestore
-Events
-1612483200
-peopleSignedDict
-Alex
-state: 1
-Mark:
-state: 0
-1606172400
-peopleSignedDict
-Users
-1zyXSMMlGiaUusUNLQWAxfsI4pM2
-name: Alex
-4zyXSMMlGiaUusUNLQWAxfsI4pM2
-name: Mark
I'm using Node.js and trying to update Events.1612483200.peopleSignedDict.Alex.state to change the state from 1 to 0. To achieve it I'm trying to do:
var randomName = getRandomDBName();
db.collection('Events').doc('1612483200').set({
'peopleSignedDict.'+randomName+'.state': 0
});
But ofc my IDE (PhpStorm) yells at me telling that this syntax is not valid, which should be the correct syntax to approach this?
There's a few way to do this.
My favorite is to first construct the key in a separate variable, and then use that in the collection:
let randomName = getRandomDBName();
let updates = {};
updates['peopleSignedDict.'+randomName+'.state'] = 0;
db.collection('Events').doc('1612483200').update(updates);
But a common (and shorter) alternative is:
let randomName = getRandomDBName();
db.collection('Events').doc('1612483200').update({
[`peopleSignedDict.${randomName}.state`]: 0;
});
I'm actually not sure if the [] are needed in that last snippet, so if you have problems with it, try it without them and let me know.
In my app, I have comments that have a field value of threadCommentCount. I want to order the comments using orderBy threadCommentCount descending and then have pagination continue this using startAfter(lastThreadCommentCount). The problem is when threadCommentCount is 0, which is a lot of them, it will return and the same data every time since it starts at 0 everytime. Here is the query:
popularCommentsQuery = db
.collection('comments')
.where('postId', '==', postId)
.orderBy('threadCommentCount', 'desc')
.startAfter(startAfter)
.limit(15)
.get()
This will return the same comments everytime once threadComment count is 0. I'm unable to send the last document snapshot because im using cloud functions and I dont want to send the documentSnapshot in a get query parameter. I don;t really care how the comments are ordered after threadCommentCount is 0, I just need to not get any duplicates. Any help is great!
All Firestore queries have an implicit orderBy("__name__", direction) to resolve any ties between documents that have the same values for the other named orderBy fields. This makes the final sort order stable. But it also enables you to pass another argument to startAfter to provide the document ID of the anchor document that you wish to use for the purpose of pagination.
.startAfter(lastThreadCommentCount, lastDocumentId)
Between these two values, you should be able to uniquely identify the document in the result set to start the next page.
so, I was trying OrderBy and StartAfter with two different fields(time,key) in firestore to establish pagination in flastList.
Key point is that we can pass document snapshot to define the query cursor [reference]
Here is how I managed to do it.
step 1: get the document Id(which is auto generated by firebase) with where() [reference]
const docRef = firestore().collection('shots').where('key','==','custom_key')
const fbDocIdGeneratedByFirebase= await docRef.get().docs[0].id;
step 2: get document snapshot with firebase generated document Id (which we got in the 1st step)
const docRef2= firestore().collection('shots').doc(fbDocIdGeneratedByFirebase)
const snapshot = await docRef2.get();
step 3: pass the snapshot got in step 2 to startAfter() so that the cursor will point there [reference]
let additionalQuery = firestore().collection('shots')
.orderBy("time", "desc")
.startAfter(snapshot)
.limit(this.state.limit)
let documentSnapshots = await additionalQuery.get(); // you know what to do next
...
Can you Improve the solution??
When I run the following query I am getting the document that matches as normal which is "LON" in this case.
But is there any way that I can make the response seperately return the values that didn't match or found, which is "BUJ" in this case. Instead of running a for loop for individual values.
ports = [
"LON",
"BUJ"
];
findDatas = async(coll, values, key) => {
let datas = await coll.find({[key] : values});
// let datas = await coll.find().where(key).in(values);
console.log(datas)
}
findDatas(airportsModel, ports, "iata_code")
In my DB I only have "LON" which mean "BUJ" is not found. So is there any way to make mongo to tell that the given values haven't been found? along with the found ones.
This code dynamically creates a $match stage to find the documents, the uses $facet to split into 2 pipelines, the first returns the documents, the second uses a $group stage created from the input array to count how many documents match each element. The result should be a document with 2 fields: documents and counts
matchdata={};
matchdata[key]={"$in":ports};
groupdata = {_id:null};
ports.forEach(function(p){
groupdata[p] = {"$sum":{"$cond":[{"$eq":["$" + key, p ]},1,0]}}
});
db.coll.aggregate([
{$match:matchdata},
{$facet:{
documents:[{$match:{}}],
counts:[{$group: groupdata},{$project:{_id:0}}]
}}
])