I am creating a webapp for users to query data from mongodb. I have front end using react and backend with nodejs hooking up mongodb.
I have included all relevant matching conditions under aggregate function. However I would like to have the effect of commenting out certain key-value pair, (by this I mean nullifying the effect of the query field condition). For example,
router.get('/getmongo', function(req, res) {
MongoClient.connect(process.env.DB_CONN, function(err, db) {
if (err) throw err;
db.aggregate([
{
$match:{
'employmentStatus':/employed|unemployed/i,
'gender':{$in:["male","female"]},
'age':{$gte:20, $lte:30}
}
},
{ "$group": {
"_id": "$employmentStatus",
"count": { "$sum": 1 }
}}
]).limit(4).toArray(function(err, docs) {
if (err) throw err;
res.send(JSON.stringify(docs))
db.close();
})
})
Let say on front end, the user does not select the option of age, which means there is no specification from the user's end on the age criteria. Usually for this case, users would comment out or remove the whole line of age query on MongoDB GUI, leaving only employmentStatus and gender as the only two criteria to query. I was looking into mongoAPI but can't find any useful tool to replicate that effect. Any idea or suggestion?
(I am using Mongo v2.3.3 by the way)
Instead of adding conditions in the Query at MongoDB side, you should build your query conditionally based on user input.
Whenever user selects the columns, build the $match clause using those columns and values. In that way, it will be quite dynamic.
Example PseudoCode:
var queryObject = {};
if(age.selected)
queryObject.add(ageCondition)
if(employmentStatus.selected)
queryObject.add(employmentStatusCondition)
if(gender.selected)
queryObject.add(genderCondition)
.....
......
You can build some logic like this in Node.js.
Related
I have a User schema with basic fields which include interests, location co-ordinates
I need to perform POST request with a specific UserId to get the results
app.post('api/users/search/:id',function(err,docs)
{ //find all the documents whose search is enabled.
//on documents returned in above find the documents who have atleast 3 common interests(req.body.interests) with the user with ':id'
// -----OR-----
//find the documents who stay within 'req.body.distance' compared to location of user with':id'
//Something like this
return User
.find({isBuddyEnabled:true}),
.find({"interests":{"$all":req.body.interests}}),
.find({"_id":req.params.id},geoLib.distance([[req.body.gcordinates],[]))
});
Basically i need to perform find inside find or Query inside query..
As per your comments in the code you want to use multiple conditions in your find query such that either one of those condition is satisfied and returns the result based on it. You can use $or and $and to achieve it. A sample code with conditions similar to yours is given below.
find({
$or:[
{ isBuddyEnabled:true },
{ "interests": { "$all":req.body.interests }},
{ $and:[
{ "_id":req.params.id },
{ geoLib.distance...rest_of_the_condition }
]
}
]
});
Question: When creating something like a simple many to many friendship in mongoose, I know how to create it on ONE object, for instance, the code below in the controller shows that I am finding one user, and pushing to his friends array another user, being referenced via ObjectId.
In this way, when I look at the Json file, I can see user with _id of "57ed2e8c9cf3083c2ccec173", has a new friend in his friend's array, and I can run a population to get that friend user document. However, user who was added as a friend does not have these capabilities because his array of friends is still empty.
I know there are multiple ways to go about this, as I have read the docs, which say I could simply now push user 1 into user 2's friends array, but, in the words of the docs: "It is debatable that we really want two sets of pointers as they may get out of sync. Instead we could skip populating and directly find() the stories we are interested in."
In other words, if you have an event model with many users, and user model with many events, and you need to access the array of users from the event document, and the array of events from the user document... Would it be best to just push each instance into each other?
Is this the correct way of thinking?
Thanks
```
app.post('/friendships', function(req, res) {
User.findOne({
_id: "57ed2e8c9cf3083c2ccec173"
}, function(err, user1) {
User.findOneAndUpdate({
_id: "57ed2ebbedcd96a4536467f7"
}, {$push: {friends: user1 }}, {upsert: true}, function(err, user2) {
console.log("success");
})
})
});
```
Yes, this is the correct way of thinking, considering the limitations of Mongo for that sort of data.
When you store such an information in two places, you need to make sure that it is consistent - i.e. either it is present in both places or not. You don't have transactions in Mongo so the only way you can do it is to chain the requests and manually roll back the first one if the second one failed, hoping that it's possible to do (which may not be the case - if the second update failed because you lost a connection to the database, there is a good chance that your rollback will fail as well, in which case your database is left in an inconsistent state).
An alternative would be to store only one half of the relationship - e.g. only store events in users, but no users in events, using your example. That way the data would be consistently stored in one place but then if you wanted to get a list of users for a certain event, you'd have to make a possibly expensive database lookup instead of having it already present in the event document.
In practice in most cases I have seen storing data in two places and trying to keep them consistent.
Though it is usually done with storing documents IDs, so instead of:
{$push: {friends: user1}}
it's usually:
{$push: {friends: user1._id}}
(or just using the _id if you have it in the first place)
And instead of $push you can use $addToSet - see: https://docs.mongodb.com/manual/reference/operator/update/addToSet/
Here is a basic concept of adding a two-directional friendship between id1 and id2:
function addFriendship(id1, id2) {
User.findOneAndUpdate({_id: id1}, {$addToSet: {friends: id2}}, err => {
if (err) {
// failure - no friendship added
} else {
// first friendship added, trying the second:
User.findOneAndUpdate({_id: id2}, {$addToSet: {friends: id1}}, err => {
if (err) {
// second friendship not added - rollback the first:
User.findOneAndUpdate({_id: id1}, {$pull: {friends: id2}}, err => {
if (err) {
// we're screwed
} else {
// rolled back - consistent state, no friendship
}
});
} else {
// success - both friendships added
}
});
}
});
}
Not pretty and not bulletproof but that's the most you can hope for with a database with no transactions where denormalized data is the norm.
(Of course friendship don't always work that way that they have to be bidirectional, but this is just an example of a pattern that is common for any many-to-many relationaship.)
Say i have the following Scheme
Product: {
Quantity: Number,
SelledQuantity: Number
}
Would it be possible to write a query where all the results returned are where Quantity=SelledQuantity?
If so, is there a way to use it when doing a populate? (Perhaps inside the match field in the opts object ?)
I use mysql connector.
yes as I understood your problem you can do this by following rest call.
http://localhost:3000/api/products?filter[where][SelledQuantity]=n
this will give you the desired results.
This question is more related to MySQL query. But you can achieve it by javascript as follows:
Product.find({}, fuction(err, products) {
if(err) throw err;
//considering products as array of product. Otherwise you can get to depth for array of product.
var filteredProducts = products.filter(function(p1) {
return p1.Quantity === p1.SelledQuantity;
});
//Your desired output
console.log(filteredProducts);
});
This will be slow but will work for smaller database size. For more optimized answer, ask the question in mysql section with respect to database and table structure.
I have a collection where I am storing 2 locations for every entry. These locations are
the starting point and destination point for a particular route. I know I can write the following query to find the nearby places to query for a particular location:
exports.nearby = function(req, res){
db.testpool.find({
fromLoc: {
$near: [28.423168, 77.045639],
$maxDistance: 1
},
}).toArray(function(err, doc){
res.send(doc);
if(err)
console.log(err+'');
else
{
res.send(doc+'');
}});
But i dont know the query which i should make in order to find the relevant routes which
not only match fromLoc but also the toLoc and i keep getting error When i do something like this :-
exports.nearby = function(req, res){
db.testpool.find({
fromLoc: {
$near: [28.423168, 77.045639],
$maxDistance: 1
},
toLoc: {
$near: [28.649573, 77.125284],
$maxDistance: 1
}
}).toArray(function(err, doc){
res.send(doc);
if(err)
console.log(err+'');
else
{
res.send(doc+'');
}
});
So basically I want to find out the most relevant car pools entered by users which will have
2 locations per entry, e.g. starting location and destination location.
Rigth now you can't have more than one geospatial index per collection*. If you want two indexes you will need at least two differents collections. MongoDB does not have joins but you can create fields that points to another document elsewhere. Those are called DBRefs and are least powerful than proper joins but can be used for similar purposes. There are strategies supported by those "pseudo-joins" in drivers of node.js like the mongoose's populate.
You will not be able to do what you want with a simple query at least until mongoDB supports multiples geospatial indexes. You will need to code it.
For example, we can have two collections. One with the routes and the other ones with locations. The one with the locations is similar to this:
{
"geo": [1,2],
"origin": [/*dbrefs array*/],
"destination": [/*dbrefs array*/]
}
Then we make two queries, one searching places near to the origin, the other one with places near the destinations. When we already have the result of both queries we do the intersection of the origin field of the first query with the destination field of the second one. To compare two objects you can use deepEqual in the assert library:
var intersection = originField.filter(function(origin){
for(var k in destinationField){
if(deepEqual(destinationField[k],origin)){
return true;
};
};
return false;
})
We have the DBRefs of the desired routes that contains it's collection ant _id so now we only needs to query those routes. Do it in a row with $in.
Hope this helps.
*Check issue 2331 in mongodb's jira.
I have a Problem in with my REST API. It will not correctly communicate with mongodb. Node.js throw following Exception:
Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters
My Manager looks like follows:
app.get('/users/:id', function(req, res) {
userProvider.fetchUserById(req.params.id, function(error, user) {
if (user == null) {
res.send(error, 404);
} else {
res.send(user);
}
});
});
My Provider looks like follows:
this.fetchUserById = function(id, cb) {
this.db.collection(usersTable, function(error, users) {
if (error) {
cb(error, null);
} else {
users.findOne({
_id:users.db.bson_serializer.ObjectID.createFromHexString(id)
}, function(error, result) {
cb(error, result);
});
}
});
};
My Mongoobject looks like follows:
{
"title": "title",
"givenname": "Vorname",
"lastname": "Nachname",
"name": "Vorname Nachname",
"username": "Benutzername",
"password": "Kennwort",
"email": "kuerzel#emailadresse.de",
"phone": "0000 - 000000",
"fax": "000000 - 000000",
"lastlogin": "15.11.2013",
"createdate": "15.11.2013",
"changedate": "15.11.2013",
"state": "1",
"_id": ObjectId("5283fbf56e3adb01f1000001")
}
When I now send a GET: /users/1 the error will thrown.
I´am new in Node.js. :-)
If your human readable IDs are unique, you can use them.
Just insert the documents with your nice id:
db.users.insert({ _id: myVeryNiceUniqueId, title: '...' }, ...);
As alternative, you could add an additional 'external id' to the document and create an unique index. But you will get many, many problems when you want to scale (sharding). Who is going to create the unique IDs? You will have something not-so-human-readable, I promise.
Edit
In Mysql I have start the Ids from 5XXXX and used them. How can I realize this with mongodb
You should ask another question or search answers. It is not a trivial task and you have to take concurrency into account. Maybe the concept of findAndModify or "Upsert" could help you. One of the most important things is that you need a way of atomic updates for the ID-generator. In one project in past I used Redis for this, but I'd always use MongoDB IDs since then. You should know that MongoDB ObjectIDs usually are generated by the Client-Driver. For REST services this is ideal: save document to DB and return status 201 Created, url=.../user/98357348957.
btw. should I generate the OrderIDs in Javascript by checking the highest value of an external id in the collection?
As said, you can get into trouble with usage of duplicate IDs when your server is in high contention. Try to find a solution with findAndModify or cache a block of IDs and use them in one process. I probably wouldn't search max and increment but use a special collection for ID-management.
What is here best practice?
Don't know. Using an ID-Generator. Depends on your enviroment (single process, single machine). If it is a simple setup, you'll find simple solutions. The setup for Twitter is not simple and they have developed a solution called twitter / snowflake. If you really want very nice, short IDs, but you have more than one process/machine, Redis might be want you want. And if the ID doesn't have to be contigous (without small gaps), you could draw a bunch of new IDs (e.g 100) and cache them for future consumption per process.
Can I hash the objectID to an 5 digit number?
No, there is no guarantee that this number is unique.