I'm trying to build an application, using MongoDB and Node.JS. I have 3 models: User, Ride, Participating.
Participating contains a userID and a rideID. It is almost as with a SQL logic: Participating links the two others models.
I'd like to, using a userID, return every Ride thanks to Participating Model
I tried to use a forEach, as the first request returns an array.
router.get('/getAllRide/:userID',function(req,res){
let userID = req.params.userID
let return = []
Participating.find({_idUser: userID })
.then(participating => {
participating.forEach(element => {
Ride.find({_id: element._id})
.exec()
.then(ride => {
retour.push(ride)})
});
res.status(200).json(return)
});
At the end of this code, the array return is empty, while it is supposed to contain every Ride whose _id is in an entity Participating.
OK, there are a couple of issues here:
return is a keyword. You probably shouldn't be using it as a variable name.
Database calls are asynchronous. forEach loops are synchronous. This means that you're immediately going to be returning retour (which looks undefined).
Mongoose has tools to populate nested relationships -- it's best not to do it in application code. Even if you are doing this in application code, it's likely best not to iterate over your results & do new finds -- instead, it's better to construct a single find query that returns all of the new documents you need.
If you did want to do this in application code, you'd want to either use async/await or Promise.all:
const toReturn = [];
const findPromises = participating.map(element => {
return Ride.find({_id: element._id})
.exec()
.then(result => toReturn.push(result)
});
return Promise.all(findPromises).then(() => res.status(200).json(toReturn));
(note: rather than using Promise.all, if you're using Bluebird you could instead use Promise.map.
Related
Checked couple of old similar questions but no luck in finding the right answers.
I have a mongo query that uses await/async to fetch results from the database, So the code looks like this :
(async () => {
//get connection object
const db = await getDatabaseConnection("admindb");
//get user data object
const configResponse = await db.collection("config").find({}).toArray();
//prints results
console.log("Successfully Fetched Configuration", configResponse);
})();
The above code works fine, just that it returns me an array of elements since I have used the .toArray() method.
Sample Result:
[
{
_id: "6028d30db7ea89f74df013d9",
tokenize: false,
configurations: { theme: {} }
}
]
Is there a NATIVE WAY (not looking forEach responses) to get result as an object since I will always have only one document returned. I have multiple queries that returns only one document, Hence accessing using the 0th index everytime did not seem the right way.
I am thinking if the findOne() will help you.
Check this out docs
await db.collection("config").findOne({});
Here are some examples
I am using Cloud Function to send a notification to mobile device. I have two collection in Firestore clientDetail and clientPersonalDetail. I have clientID same in both of the collection but the date is stored in clientDetail and name is stored in clientPersonal.
Take a look:
ClientDetail -- startDate
-- clientID
.......
ClientPersonalDetail -- name
-- clientID
.........
Here is My full Code:
exports.sendDailyNotifications = functions.https.onRequest( (request, response) => {
var getApplicants = getApplicantList();
console.log('getApplicants', getApplicants);
cors(request, response, () => {
admin
.firestore()
.collection("clientDetails")
//.where("clientID", "==", "wOqkjYYz3t7qQzHJ1kgu")
.get()
.then(querySnapshot => {
const promises = [];
querySnapshot.forEach(doc => {
let clientObject = {};
clientObject.clientID = doc.data().clientID;
clientObject.monthlyInstallment = doc.data().monthlyInstallment;
promises.push(clientObject);
});
return Promise.all(promises);
}) //below code for notification
.then(results => {
response.send(results);
results.forEach(user => {
//sendNotification(user);
});
return "";
})
.catch(error => {
console.log(error);
response.status(500).send(error);
});
});
}
);
Above function is showing an object like this
{clienId:xxxxxxxxx, startDate:23/1/2019}
But I need ClientID not name to show in notification so I'll have to join to clientPersonal collection in order to get name using clientID.
What should do ?
How can I create another function which solely return name by passing clientID as argument, and waits until it returns the name .
Can Anybody please Help.?
But I need ClientID not name to show in notification so I'll have to join to clientPersonal collection in order to get name using clientID. What should do ?
Unfortunately, there is no JOIN clause in Firestore. Queries in Firestore are shallow. This means that they only get items from the collection that the query is run against. There is no way to get documents from two top-level collection in a single query. Firestore doesn't support queries across different collections in one go. A single query may only use properties of documents in a single collection.
How can I create another function which solely return name by passing clientID as argument, and waits until it returns the name.
So the most simple solution I can think of is to first query the database to get the clientID. Once you have this id, make another database call (inside the callback), so you can get the corresponding name.
Another solution would be to add the name of the user as a new property under ClientDetail so you can query the database only once. This practice is called denormalization and is a common practice when it comes to Firebase. If you are new to NoQSL databases, I recommend you see this video, Denormalization is normal with the Firebase Database for a better understanding. It is for Firebase realtime database but same rules apply to Cloud Firestore.
Also, when you are duplicating data, there is one thing that need to keep in mind. In the same way you are adding data, you need to maintain it. With other words, if you want to update/detele an item, you need to do it in every place that it exists.
The "easier" solution would probably be the duplication of data. This is quite common in NoSQL world.
More precisely you would add in your documents in the ClientDetail collection the value of the client name.
You can use two extra functions in this occasion to have your code clear. One function that will read all the documents form the collection ClientDetail and instead of getting all the fields, will get only the ClientID. Then call the other function, that will be scanning all the documents in collection ClientPersonalDetail and retrieve only the part with the ClientID. Compare if those two match and then do any operations there if they do so.
You can refer to Get started with Cloud Firestore documentation on how to create, add and load documents from Firestore.
Your package,json should look something like this:
{
"name": "sample-http",
"version": "0.0.1",
"dependencies": {
"firebase-admin": "^6.5.1"
}
}
I have did a little bit of coding myself and here is my example code in GitHub. By deploying this Function, will scan all the documents form one Collection and compare the ClientID from the documents in the other collection. When it will find a match it will log a message otherwise it will log a message of not matching IDs. You can use the idea of how this function operates and use it in your code.
I have a scenario in which I need to insert multiple records. I have a table structure like id (it's fk from other table), key(char), value(char). The input which needs to be saved would be array of above data. example:
I have some array objects like:
lst = [];
obj = {};
obj.id= 123;
obj.key = 'somekey';
obj.value = '1234';
lst.push(obj);
obj = {};
obj.id= 123;
obj.key = 'somekey1';
obj.value = '12345';
lst.push(obj);
In MS SQL, I would have created TVP and passed it. I don't know how to achieve in postgres.
So now what I want to do is save all the items from the list in single query in postgres sql, using pg-promise library. I'm not able to find any documentation / understand from documentation. Any help appreciated. Thanks.
I am the author of pg-promise.
There are two ways to insert multiple records. The first, and most typical way is via a transaction, to make sure all records are inserted correctly, or none of them.
With pg-promise it is done in the following way:
db.tx(t => {
const queries = lst.map(l => {
return t.none('INSERT INTO table(id, key, value) VALUES(${id}, ${key}, ${value})', l);
});
return t.batch(queries);
})
.then(data => {
// SUCCESS
// data = array of null-s
})
.catch(error => {
// ERROR
});
You initiate a transaction with method tx, then create all INSERT query promises, and then resolve them all as a batch.
The second approach is by concatenating all insert values into a single INSERT query, which I explain in detail in Performance Boost. See also: Multi-row insert with pg-promise.
For more examples see Tasks and Transactions.
Addition
It is worth pointing out that in most cases we do not insert a record id, rather have it generated automatically. Sometimes we want to get the new id-s back, and in other cases we don't care.
The examples above resolve with an array of null-s, because batch resolves with an array of individual results, and method none resolves with null, according to its API.
Let's assume that we want to generate the new id-s, and that we want to get them all back. To accomplish this we would change the code to the following:
db.tx(t => {
const queries = lst.map(l => {
return t.one('INSERT INTO table(key, value) VALUES(${key}, ${value}) RETURNING id',
l, a => +a.id);
});
return t.batch(queries);
})
.then(data => {
// SUCCESS
// data = array of new id-s;
})
.catch(error => {
// ERROR
});
i.e. the changes are:
we do not insert the id values
we replace method none with one, to get one row/object from each insert
we append RETURNING id to the query to get the value
we add a => +a.id to do the automatic row transformation. See also pg-promise returns integers as strings to understand what that + is for.
UPDATE-1
For a high-performance approach via a single INSERT query see Multi-row insert with pg-promise.
UPDATE-2
A must-read article: Data Imports.
How can I know the count of a model that data has been saved? there is a method of Model.count(), but it doesn't seem to work.
var db = mongoose.connect('mongodb://localhost/myApp');
var userSchema = new Schema({name:String,password:String});
userModel =db.model('UserList',userSchema);
var userCount = userModel.count('name');
userCount is an Object, which method called can get a real count?
Thanks
The reason your code doesn't work is because the count function is asynchronous, it doesn't synchronously return a value.
Here's an example of usage:
userModel.count({}, function( err, count){
console.log( "Number of users:", count );
})
The code below works. Note the use of countDocuments.
var mongoose = require('mongoose');
var db = mongoose.connect('mongodb://localhost/myApp');
var userSchema = new mongoose.Schema({name:String,password:String});
var userModel =db.model('userlists',userSchema);
var anand = new userModel({ name: 'anand', password: 'abcd'});
anand.save(function (err, docs) {
if (err) {
console.log('Error');
} else {
userModel.countDocuments({name: 'anand'}, function(err, c) {
console.log('Count is ' + c);
});
}
});
You should give an object as argument
userModel.countDocuments({name: "sam"});
or
userModel.countDocuments({name: "sam"}).exec(); //if you are using promise
or
userModel.countDocuments({}); // if you want to get all counts irrespective of the fields
For the older versions of mongoose, use
userModel.count({name: "sam"});
The collection.count is deprecated, and will be removed in a future version. Use collection.countDocuments or collection.estimatedDocumentCount instead.
userModel.countDocuments(query).exec((err, count) => {
if (err) {
res.send(err);
return;
}
res.json({ count: count });
});
Background for the solution
As stated in the mongoose documentation and in the answer by Benjamin, the method Model.count() is deprecated. Instead of using count(), the alternatives are the following:
Model.countDocuments(filterObject, callback)
Counts how many documents match the filter in a collection. Passing an empty object {} as filter executes a full collection scan. If the collection is large, the following method might be used.
Model.estimatedDocumentCount()
This model method estimates the number of documents in the MongoDB collection. This method is faster than the previous countDocuments(), because it uses collection metadata instead of going through the entire collection. However, as the method name suggests, and depending on db configuration, the result is an estimate as the metadata might not reflect the actual count of documents in a collection at the method execution moment.
Both methods return a mongoose query object, which can be executed in one of the following two ways. Use .exec() if you want to execute a query at a later time.
The solution
Option 1: Pass a callback function
For example, count all documents in a collection using .countDocuments():
someModel.countDocuments({}, function(err, docCount) {
if (err) { return handleError(err) } //handle possible errors
console.log(docCount)
//and do some other fancy stuff
})
Or, count all documents in a collection having a certain name using .countDocuments():
someModel.countDocuments({ name: 'Snow' }, function(err, docCount) {
//see other example
}
Option 2: Use .then()
A mongoose query has .then() so it’s “thenable”. This is for a convenience and query itself is not a promise.
For example, count all documents in a collection using .estimatedDocumentCount():
someModel
.estimatedDocumentCount()
.then(docCount => {
console.log(docCount)
//and do one super neat trick
})
.catch(err => {
//handle possible errors
})
Option 3: Use async/await
When using async/await approach, the recommended way is to use it with .exec() as it provides better stack traces.
const docCount = await someModel.countDocuments({}).exec();
Learning by stackoverflowing,
Using mongoose.js you can count documents,
count all
const count = await Schema.countDocuments();
count specific
const count = await Schema.countDocuments({ key: value });
The highest voted answers here are perfectly fine I just want to add up the use of await so that the functionality asked for can be achieved:
const documentCount = await userModel.count({});
console.log( "Number of users:", documentCount );
It's recommended to use countDocuments() over 'count()' as it will be deprecated going on. So, for now, the perfect code would be:
const documentCount = await userModel.countDocuments({});
console.log( "Number of users:", documentCount );
Model.count() method is deprecated in mongoose version 6.2.0. If you want to count the number of documents in a collection, e.g. count({}), use the estimatedDocumentCount() function instead. Otherwise, use the countDocuments() function instead.
Model.estimatedDocumentCount() Estimates the number of documents in the MongoDB collection. It is Faster than using countDocuments() for large collections because estimatedDocumentCount() uses collection metadata rather than scanning the entire collection.
Example:
const numAdventures = await Adventure.estimatedDocumentCount();
reference : https://mongoosejs.com/docs/api.html#model_Model.estimatedDocumentCount
As said before, your code will not work the way it is. A solution to that would be using a callback function, but if you think it would carry you to a 'Callback hell', you can search for "Promisses".
A possible solution using a callback function:
//DECLARE numberofDocs OUT OF FUNCTIONS
var numberofDocs;
userModel.count({}, setNumberofDocuments); //this search all DOcuments in a Collection
if you want to search the number of documents based on a query, you can do this:
userModel.count({yourQueryGoesHere}, setNumberofDocuments);
setNumberofDocuments is a separeted function :
var setNumberofDocuments = function(err, count){
if(err) return handleError(err);
numberofDocs = count;
};
Now you can get the number of Documents anywhere with a getFunction:
function getNumberofDocs(){
return numberofDocs;
}
var number = getNumberofDocs();
In addition , you use this asynchronous function inside a synchronous one by using a callback, example:
function calculateNumberOfDoc(someParameter, setNumberofDocuments){
userModel.count({}, setNumberofDocuments); //this search all DOcuments in a Collection
setNumberofDocuments(true);
}
NodeJS + Express, MongoDB + Mongoose
I have a JSON feed where each record has a set of "venue" attributes (things like "venue name" "venue location" "venue phone" etc). I want to create a collection of all venues in the feed -- one instance of each venue, no dupes.
I loop through the JSON and test whether the venue exists in my venue collection. If it doesn't, save it.
jsonObj.events.forEach(function(element, index, array){
Venue.findOne({'name': element.vname}, function(err,doc){
if(doc == null){
var instance = new Venue();
instance.name = element.vname;
instance.location = element.location;
instance.phone = element.vphone;
instance.save();
}
}
}
Desired: A list of all venues (no dupes).
Result: Plenty of dupes in the venue collection.
Basically, the loop created a new Venue record for every record in the JSON feed.
I'm learning Node and its async qualities, so I believe the for loop finishes before even the first save() function finishes -- so the if statement is always checking against an empty collection. Console.logging backs this claim up.
I'm not sure how to rework this so that it performs the desired task. I've tried caolan's async module but I can't get it to help. There's a good chance I'm using incorrectly.
Thanks so much for pointing me in the right direction -- I've searched to no avail. If the async module is the right answer, I'd love your help with how to implement it in this specific case.
Thanks again!
Why not go the other way with it? You didn't say what your persistence layer is, but it looks like mongoose or possibly FastLegS. In either case, you can create a Unique Index on your Name field. Then, you can just try to save anything, and handle the error if it's a unique index violation.
Whatever you do, you must do as #Paul suggests and make a unique index in the database. That's the only way to ensure uniqueness.
But the main problem with your code is that in the instance.save() call, you need a callback that triggers the next iteration, otherwise the database will not have had time to save the new record. It's a race condition. You can solve that problem with caolan's forEachSeries function.
Alternatively, you could get an array of records already in the Venue collection that match an item in your JSON object, then filter the matches out of the object, then iteratively add each item left in the filtered JSON object. This will minimize the number of database operations by not trying to create duplicates in the first place.
Venue.find({'name': { $in: jsonObj.events.map(function(event){ return event.vname; }) }}, function (err, docs){
var existingVnames = docs.map(function(doc){ return doc.name; });
var filteredEvents = jsonObj.events.filter(function(event){
return existingVnames.indexOf(event.vname) === -1;
});
filteredEvents.forEach(function(event){
var venue = new Venue();
venue.name = event.vname;
venue.location = event.location;
venue.phone = event.vphone;
venue.save(function (err){
// Optionally, do some logging here, perhaps.
if (err) return console.error('Something went wrong!');
else return console.log('Successfully created new venue %s', venue.name);
});
});
});