I am trying to query an old mongoDB collection in node.js with the native driver.
It uses Legacy UUID as its _id field.
In my code I am using Binary values such as: "2u0kLwUZuEWQvjqOjgQU4g=="
and I want to query the collection with find() operation.
When trying to test it directly with mongoDB I am able to use the following query:
db.getCollection('myCollection').find({_id: BinData(3, "2u0kLwUZuEWQvjqOjgQU4g==")})
to find my item.
But what I need to do is to find multiple items.
So I need to use something like this:
db.getCollection('myCollection').find({_ids: { $in: [BinData(3, "2u0kLwUZuEWQvjqOjgQU4g=="), BinData(3, "3u0kLwUZuEWQvjqOjgQU4g==")...]}})
but it does not seem to work and always returns zero records.
I am not sure why? and what might be the correct way to query multiple Legacy UUIDs?
Related
I'm using an ajax request from the front end to load more comments to a post from the back-end which uses NodeJS and mongoose. I won't bore you with the front-end code and the route code, but here's the query code:
Post.findById(req.params.postId).populate({
path: type, //type will either contain "comments" or "answers"
populate: {
path: 'author',
model: 'User'
},
options: {
sort: sortBy, //sortyBy contains either "-date" or "-votes"
skip: parseInt(req.params.numberLoaded), //how many are already shown
limit: 25 //i only load this many new comments at a time.
}
}).exec(function(err, foundPost){
console.log("query executed"); //code takes too long to get to this line
if (err){
res.send("database error, please try again later");
} else {
res.send(foundPost[type]);
}
});
As was mentioned in the title, everything works fine, my problem is just that this is too slow, the request is taking about 1.5-2.5 seconds. surely mongoose has a method of doing this that takes less to load. I poked around the mongoose docs and stackoverflow, but didn't really find anything useful.
Using skip-and-limit approach with mongodb is slow in its nature because it normally needs to retrieve all documents, then sort them, and after that return the desired segment of the results.
What you need to do to make it faster is to define indexes on your collections.
According to MongoDB's official documents:
Indexes support the efficient execution of queries in MongoDB. Without indexes, MongoDB must perform a collection scan, i.e. scan every document in a collection, to select those documents that match the query statement. If an appropriate index exists for a query, MongoDB can use the index to limit the number of documents it must inspect.
-- https://docs.mongodb.com/manual/indexes/
Using indexes may cause increased collection size but they improve the efficiency a lot.
Indexes are commonly defined on fields which are frequently used in queries. In this case, you may want to define indexes on date and/or vote fields.
Read mongoose documentation to find out how to define indexes in your schemas:
http://mongoosejs.com/docs/guide.html#indexes
While using Meteor, I sometimes access the underlying Node Mongo driver so I can make bulk updates and inserts.
const bulk = Coll.rawCollection().initializeOrderedBulkOp();
bulk.insert({key_id: Mongo.Collection.ObjectID()}); // note key_id is an ObjectID
...
bulk.execute();
But the value of the key_id fields ends up being the plain subdocument {_str: '...'} when I look in the database after the insert.
Is there any way to use bulk operations in Node's Mongo library (whatever it is Meteor uses) and keep ObjectID's as Mongo's ObjectID type?
(There's many posts about the nature of the different ID types, and explaining Minimongo, etc. I'm interested specifically about the bulk operations converting ObjectID's into plain objects, and solving that issue.)
From Neil's top-level comment
On a native method you would actually need to grab the native implementation. You should be able to access from the loaded driver through MongoInternals [...]
Mongo.Collection.ObjectID is not a plain ObjectId representation, and is actually a complex object for Meteor internal use. Hence why the native methods don't know how to use the value.
So if you have some field which is an ObjectId, and you're using some method of a Meteor Collection's rawCollection (for example,
.distinct
.aggregate
.initializeOrderedBulkOp
.initializeUnorderedBulkOp
), you'll want to convert your ObjectId's using
const convertedID = new MongoInternals.NpmModule.ObjectID(
originalID._str
);
// then use in one of the arguments to your function or something
const query = {_id: convertedID};
before calling the method on them.
TLDR
Is there a way to limit queryByExample to a collection in NodeJS?
Problem faced
I have a complex query with some optional fields (i.e. sometimes some search fields will be omitted). So I need to create a query dynamically, e.g. in JSON. QueryByExample seems to be the right tool to use here as it gives me that flexibility to pass a JSON. However my problem is that I would like to limit my search to only one collection or directory.
e.g. I was hoping for something like
searchJSON = {
title: { $word: "test" },
description: { $word: "desc" }
};
//query
db.documents.query(qb.where(
qb.collection("collectionName"),
qb.byExample(searchJSON)
)).result()...
In this case searchJSON could have been built dynamically, for example maybe sometimes title may be omitted from the search.
This doesn't work because the query builder only allows queryByExample to be the only query. But I'd instead like to built a dynamic search query which is limited to a collection or directory.
At present, I think you would have to express the query with QueryBuilder instead of Query By Example using
qb.and([
qb.collection('collectionName'),
qb.word('title', 'test'),
qb.word('description', 'desc')
])
See http://docs.marklogic.com/jsdoc/queryBuilder.html#word
That said, it should be possible for the Node.js API to relax that restriction based on the fixes in MarkLogic 9.0-2
Please file an issue on https://github.com/marklogic/node-client-api
Is there a way to tell the native MongoDB driver for NodeJS to automatically convert the contents of an _id field into an ObjectID?
Say, in this situation:
db.collection("collection").updateOne({_id: data._id}, data)
It's not that data._id = ObjectID(data.id) is hard, but it's another thing to miss each and every time.
There is no way to do that natively. You can make some function for wrapping your mongo queries where you will check params and if it's "_id" parse it to ObjectId.
I come from MySQL world so mongo queries are a bit difficult to make considering I can't really make sense of mongo style queries. I am trying to make a query for finding a string. The problem is from my very primitive knowledge about mongodb queries, the query I made isn't working. I tried it in mongoose as well in mongo shell.
Schema:
mongoose.Schema({
doctorID : String,
patientIDList : Array // array of strings
});
Query Objective:
I want to find a doctor with doctorID and then look inside the patientIDList for an ID xxx. If the patientIDList doesn't contains xxx then add xxx in the list otherwise just add nothing.
Query:
The 2 queries I tried
MyModel.findOne({'doctorID':newAppointment.doctorID}, {'patientIDList' : newAppointment.patientID}, function(err){...});
MyModel.findOne({'doctorID': newAppointment.doctorID, 'patientIDList': newAppointment.patientID}, function(err){...});
What am I doing wrong? How can I make a query?
It's always a bit of challenge to switch from a SQL to NoSQL DB and other way around. What you are trying to do is check if a value exists in an array. If the array is a string array you can simply query for the value in array.
MyModel.findOne({doctorID : newAppointment.doctorID}, {patientIDList :newAppointment.doctorID}, function(err, res){
console.log(err, res);
})
Further read: https://docs.mongodb.com/manual/tutorial/query-documents/#match-an-array-element
Relevant Question: Find document with array that contains a specific value