I have a function running in a node app that I can't get to work due to my lack of understanding on how to write asynchronous code properly. Below is a function that takes in a profile with emails. I would like to loop through each email and check to see whether that user exists in my database. If they do, I would like to return the callback given and completely exist the function without doing anything else. If the user is not found, I would then like to create a new user based on the information given in the profile, and then return the same callback with the newly created user. As of now, the function works as intended, except that it creates a new user even when a user is already found in my database. (the 'User' variable is defined above and has the 'create' function. Also, I would like to avoid using the 'async' node module if at all possible)
function processProfile(profile, callback) {
var existingUser;
if (profile.emails) {
profile.emails.forEach(function(email) {
console.log("Searching for user with this email:" + email.value);
existingUser = findUserByEmail(email.value);
if (existingUser) {
console.log("Found the existing user");
return callback(null, existingUser);
}
});
if(!existingUser){
console.log("Creating new user");
var newUser = {
id: profile.id,
firstName: profile.name.givenName,
lastName: profile.name.familyName,
email: profile.emails[0].value
};
user.create(newUser, profile.provider, function(err, user) {
if (err) throw err;
return callback(null, user);
});
}
}
}
Is there something wrong with this?
function processProfile(profile, callback) {
var existingUser;
var index = 0;
function processNextEmail() {
if(index >= profile.emails.size()) return; //When we've popped nothing exit
var email = profile.emails[index++];
console.log("Searching for user with this email:" + email.value);
existingUser = findUserByEmail(email.value);
if (existingUser) {
console.log("Found the existing user");
callback(null, existingUser);
processEmail();//recursive call to prcess the next email
} else {
console.log("Creating new user");
var newUser = {
id: profile.id,
firstName: profile.name.givenName,
lastName: profile.name.familyName,
email: profile.emails[0].value
};
user.create(newUser, provider, function(err, user) {
if (err) throw err;
callback(null, user);
processNextEmail();//recursive call to process the next email after creating the user and adding it to the database.
});
}
}
processNextEmail();
}
If you need the recursive logic to not remove the emails, you can do a simple modification involving an indice within the scope of the processProfile() closure.
Also note, the return callback() lines, don't really do anything. Returning from functions that are happening asynchronously is a waste of time. Just call the callback, and then you can call an empty return to skip the rest of the function if you wish, but it is unecessary, unless the return effects the flow of logic.
EDIT: It turns out this is example is too simple to be much more interesting. The code below I used as an example for some people at work who were having trouble grasping async code. The one time I think that it is okay to use sync code in node is for gathering configuration data. Let's pretend that we have configuration stored in a file, that in turn gets the filename from that file, and gathers another layer of configuration data from another file. We can do this two ways, using readFileSyn, or using readFile. The asynchronous version is tricky, because we need to wait for the first step to complete, because we have to grab the filename from the first file, in order to know where the second file is stored. Below is the code for both the sync solution and async solution.
//The synchronous way
function useConfigurationData(configData) {
dosomethinginterestingwith(configData);
}
function getConfigurationData(fileName) {
var fileName2 = fs.readFileSync(fileName);
var configurationData = fs.readFileSync(fileName2);
return configurationData;
}
var fileData = getConfigurationData('someFile');
useConfigurationData(fileData);
//The equivalent async way
function getConfigurationData(fileName, useConfigDataCallBack) {
fs.readFile(fileName, getConfigDataStepTwo);
function getConfigDataStepTwo(err, fileName2) {
fs.readFile(fileName2, getConfigDataStepThree);
}
function getConfigDataStepThree(err, fileData) {
useConfigDataCallBack(fileData);
}
}
getConfigurationData('someFile', useConfigurationData);
Notice that the callback we supply to getConfigurationData is the last step. We could also just rely on the globally defined getConfigurationData function, but passing it in as a callback is better style.
What I like about this syntax, is the code withing the second getConfigurationData function reads in order, very synchronously. But if you follow the flow of logic, it is all being run async. It's easy to read and follows nodes async I/O model. IN the case of configuration data, I think the synchronous option is acceptable, but this is still a good demo of how to get synchronous behavior and syntax(ish) from asynchronous callbacks.
Related
With the following code I'm not able to authenticate with a MongoDB database, which already has a Users schema and users associated with it and I was wondering how I would make sure that auth returned isAuth?:
exports.auth = function(username, password, session) {
User.findOne({username: username}, function(err, data) {
if (err) {
console.log(err);
}
var isAuth = username === data['username'] & password === data['password'];
if (isAuth) {
session.isAuthenticated = isAuth;
session.user = {username: username};
}
return isAuth;
});
};
First of all, as others have already pointed out in the comments, you shouldn't implement your own authentication logic if you don't know what you're doing. You can use Passport for that.
Now, to the code you provided. There are several problems here.
The first thing that comes to mind is that you use:
var isAuth = username === data['username'] & password === data['password'];
instead of:
var isAuth = username === data['username'] && password === data['password'];
But this is just a typo. Now, to more fundamental stuff.
You cannot return the isAuth variable because who are you going to return it to? If you think that it will get returned to the caller of exports.auth then you're wrong - the exports.auth() will return long before the return isAuth; is ever run.
Also, if yu check for error with if (err) then put the code that should be run in the case of success in the else block o otherwise it will also be run on error with undefined variables that may crash your program.
You need to either add an additional argument to your function which is a callback:
exports.auth = function(username, password, session, callback) {
User.findOne({username: username}, function(err, data) {
if (err) {
console.log(err);
callback(err);
} else {
var isAuth = username === data.username && password === data.password;
if (isAuth) {
session.isAuthenticated = isAuth;
session.user = {username: username};
}
callback(null, isAuth);
}
});
};
or to return a promise from your exports.auth function (but directly from your exports.auth function, not some other callback inside).
Using the above version you can call it with:
auth(username, password, session, function (isAuth) {
// you have your isAuth here
});
The other option would be to use promises. You can see some other answers where I explain the difference between callbacks and promises and how to use them together in more detail, which may be helpful to you in this case:
A detailed explanation on how to use callbacks and promises
Explanation on how to use promises in complex request handlers
An explanation of what a promise really is, on the example of AJAX requests
But first you need to get comfortable with callbacks.
Also, never store the passwords in cleartext in the database. Seriously, use some other solution that works like Passport. I wrote the answer to explain the process of using callbacks, not to endorse the idea of using authentication in that particular way. You have been warned.
I'm trying to send my node server multiple entries in order to update every person inside the database.
I thought a way to do it would be to loop through each person through their unique Ids and save the updated information based on this.
However, inside the Employee.findById function I cannot access the value of [i] so I cannot get the relevant employee. Running this line of code when trying to modify 2 Employees var i will output;
router.patch('/edit', function(req, res, next) {
for (var i = 0; i < req.body.length; i++) {
console.log("outside " + i)
Employee.findById(req.body[i].employeeId, function(err, employee) {
console.log("inside " + i)
/*
employee = new Employee({
name: req.body[i].name,
})
employee.save();
*/
})
}
})
I'm not sure why the console.log("inside " + i) isn't outputting the same number as the outside log?
Also, I'm not sure if the approach I am taking is the correct way of doing this?
Thanks for any advice!
When your /edit route is called, the for loop runs, logs out the "outside" statement and calls the Employee.findById method for each element in req.body, which will execute and call the "inside" console log statement when the find operation has finished. Note that this is an asynchronous operation.
The i variable does not change inside the callback of the Employee.findById call. This is because the for loop already incremented i to the number of elements in req.body.
By doing this, you will not be able to tell if all operations finished or an error occured while saving the employee documents and forward this information to the client in the response.
I would suggest using async or another flow controll library.
Your example using async:
router.patch('/edit', function(req, res, next) {
async.each(req.body, function(obj, cb) {
Employee.findById(obj.employeeId, function(err, employee) {
if (err) {
return cb(err);
}
employee = new Employee({name: obj.name});
employee.save(cb);
});
}, function (err) {
// any errors from callback will be handled here
// inform you user in the response or whatever
});
});
It's because the function findById() is asynchronous. The iteration of the loop doesn't wait findById() terminated the query.
I suggest you in order to fix this problem to use the lib Async.
It's provide some useful methods for doing things in asynchronous and wait the response.
In your case something like that can fix the problem:
async.each(req.body, function (value, callback) {
Employee.findById(value.employeeId, function(err, employee) {
employee = new Employee({
name: value.name,
})
// The callback indicate that this iteration is terminated.
employee.save(callback);
});
}, function (err) {
// When all callback() are triggered we pass here, and this tells we finished all operation.
});
I suggest you to read carefully the documentation of each
I'm developing my first test application on NodeJS and encountered the following problem: i can't figure out how to properly organize routes in ExpressJS framework. For example i want to do registration, so i create route like:
app.get('/registration', function (request, response) {
if (request.body.user.email && request.body.user.password) {
var user = new User();
var result = user.createNew(request.body.user.email, request.body.user.email);
// do stuff...
}
response.render('registration.html', config);
});
User function looks like this (not the final):
function User() {
var userSchema = new mongoose.Schema({
'email': {
'type': String,
'required': true,
'lowercase': true,
'index': { 'unique': true }
},
'password': {
'type': String,
'required': true
}
});
var userModel = mongoose.model('users', userSchema);
this.createNew = function(email, password) {
var new_user = new users({'email': email, 'password': password});
new_user.save(function(err){
console.log('Save function');
if (err)
return false;
return true;
});
}
}
I try to do a bit of structured applications like MVC. The problem is that the save method is asynch and each time i registrate new user get registration.html without waiting for the result.
Basically i need to run route callback in save callback, but how to do this in the right way i can't figure out by myself...
this.createNew = function(email, password, callback) {
var new_user = new users({'email': email, 'password': password});
new_user.save(function(err){
console.log('Save function');
if (err)
// return false;
callback (false);
else
//return true;
callback (true);
});
}
I find that whenever I'm using some module (db for example) and it's using a callback, I will often have to also use a callback for any function I wrap around it (unless I don't care about the results).
Here:
app.get('/registration', function (request, response) {
if (request.body.user.email && request.body.user.password) {
var user = new User();
// var result = user.createNew(request.body.user.email, request.body.user.email);
user.createNew(request.body.user.email, request.body.user.email, function (results) {
// do something with results (will be true or false)
// not sure why you have this with the actual registration stuff,
// but it's your site. :)
response.render('registration.html', config);
});
}
});
Also, you might want to put your object methods in the prototype, instead of:
this.createNew = function (....) {}
Try:
User.prototype.createNew = function ( ... ) { }
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Closures#Performance_considerations for info on why.
For a starting point on organising your Node.js app, try reading through the source code on this demo express.js app. Click here for the link
It's a fairly popular repo, which I often find myself referencing when building Node.js apps from scratch.
It may be a good reference for your project given it's done in an the MVC style and it uses mongoose. The routes are organised into a single file which can be found in config/routes.js. You should also take a look at the models in app/models/ for an alternative way to organise your user model
I have folowing script
var email_list = ['email1#email.com', 'email2#email.com',....'email100#email.com'];
for(i=0;i<email_list.length;i++){
if(checkEmail(email_list[i])){
//do processing save in db and email to email addresses.
}
}
This code will be blocking in nodejs how to make this non blocking?
You can do this without blocking the event loop at all, by using a recursive loop. This way what you end up with is only launching one database worker per call, at a give time. Assuming the database work you were doing was asynchronous, your code didn't really block the event loop. But the foor loop still launched a bunch of workers simultaneously, which will tend to clog the event loop(not block it). And you are right in that it is blocking the event loop while your for loop is counting from 0, to whatever the size of your array is. The following does exactly the same thing, but you only launch one database worker at a time(good), and you never count from 0 to length. Each worker is popped off the list after the work on the current email is done, and your global event loop is left to process other things, not email_list.length database requests simultaneously.
var email_list = ['email1#email.com', 'email2#email.com', 'email100#email.com'];
function checkEmailList(emails, emailCallBack, completionCallback) {
var someDataCollectdOverAllEmails = '';
function checkEmailAsync(email) {
db.doSomeDBWorkAsync(email, function (data) {
someDataCollectdOverAllEmails += data;
if (email_list.length) {
checkEmail(email_list.pop()); //If there are still emails to be checked, check the next one ine line
} else {
completionCallback(someDataCollectdOverAllEmails);//IF not, call the completionCallBack
}
emailCallBack(data);
});
}
checkEmailAsync(emails.pop());
}
function logIndividualEmailData(data) {
console.log('Sningle Email: ' + data);
}
function logGlobalEmailData(data) {
console.log('All Email Data: ' + data);
}
checkEmailList(email_list, logIndividualEmailData, logGlobalEmailData);
Process.nextTick example
process.nextTick(function () {
'use strict';
console.log('printed second');
while (true);
});
process.nextTick(function () {
'use strict';
console.log('never printed');
});
console.log('printed first');
Note however that in the example below, despite the fact that loopForever will run forever, it still allows both of our files to be read out. If we just had while(true) it would of course block and not allow this and one of our files data would not be printed out.
var files = ['blah.js', 'file.js'];
for(var i = 0; i < files.length; i++) {
fs.readFile(files[i], function (err, data) {
console.log('File data' + data);
function loopForver(loop) {//asynchronously loop forever, pretty cool, but only useful for really specific situations!
process.nextTick(function () {
if(loop) {
console.log('looping');
loopForver(true);
}
});
}
loopForver(true);
});
}
If I need to do stuff after the emails all send, I use the async library (docs), which provides some useful functions for control flow.
You will still need to rewrite checkEmail(email) into checkEmail(email, callback) as #S.D. suggests. In checkEmail you will want to call callback after everything is completed. This probably means that you will nest callbacks, calling the second async thing (sending the email) only after the first (db query) has completed successfully.
I also suggest that you follow convention by using the first callback argument as an err parameter. If you callback(null) you are explicitly saying 'there was no error'. #S.D.'s solution suggests instead callback(ok) which is the opposite of convention.
Here is an example showing a couple nested asynchronous functions and the async library.
edit - use async.eachLimit instead of async.each so you don't execute all 100 calls simultaneously
(function main(){
var emails = ["a#b", "c#d"];
var async = require('async');
async.eachLimit(
emails // array to iterate across
,10 // max simultaneous iterations
,checkEmail // an asynchronous iterator function
,function(err){ // executed on any error or every item successful
console.log('Callback of async.eachLimit');
if(err){
console.log('Error: '+err)
} else {
console.log('All emails succeeded');
};
}
);
console.log('Code below the async.eachLimit call will continue executing after starting the asynchronous jobs');
})();
function checkEmail(email, callback){
fetchFromDb(email, function(err, obj){
if(err){ return callback(err) };
sendEmail(email, function(err, obj){
if(err){ return callback(err)};
console.log('Both fetchFromDb and sendEmail have completed successfully for '+email);
callback(null);
});
});
};
function fetchFromDb(email, callback){
process.nextTick(function(){ // placeholder, insert real async function here
callback(null);
});
};
function checkEmail(email, callback){
process.nextTick(function(){ // placeholder, insert real async function here
callback(null);
});
};
I am little bit confused with my code it's not worked synchronusly as it's should be. I use everyauth to do authentication.
registerUser(function(newUserAttrs) {
var login = newUserAttrs[this.loginKey()];
user.CreateNewUser(newUserAttrs.login, newUserAttrs.password, newUserAttrs.email, function(res, err) {
if(!err) {
return usersByLogin[login] = newUserAttrs;
}
else {
throw err;
}
});
})
in another file I have write this code
exports.CreateNewUser = function(login, pass, mail, callback) {
var sql = "insert into `user`(login,mail,pass) values(?,?,?)";
client.query(sql, [login, mail, pass], function(err, results, fields) {
if(!err) {
callback(results);
console.log('test')
}
else {
callback(results, err);
}
});
};
This code are working fine. I have tested him. the only problem is they are working synchronosly (as normal). Can someone explain me what thing I have done in wrong way that make it async. I want to get it done in sync way.
The current code give me error (it's make a entry in database and produce error on browser)
Error: Step registerUser of `password` is promising: userOrErrors ; however, the step returns nothing. Fix the step by returning the expected values OR by returning a Promise that promises said values.
at Object.exec (E:\proj\Node\node_modules\everyauth\lib\step.js:68:11)
at E:\proj\Node\node_modules\everyauth\lib\stepSequence.js:26:38
at [object Object].callback (E:\proj\Node\node_modules\everyauth\lib\promise.js:13:12)
at RouteTriggeredSequence._bind (E:\proj\Node\node_modules\everyauth\lib\stepSequence.js:25:20)
at RouteTriggeredSequence.start (E:\proj\Node\node_modules\everyauth\lib\stepSequence.js:52:33)
at RouteTriggeredSequence.routeHandler (E:\proj\Node\node_modules\everyauth\lib\routeTriggeredSequence.js:13:13)
at Object.<anonymous> (native)
at nextMiddleware (E:\proj\Node\node_modules\connect\lib\middleware\router.js:175:25)
at param (E:\proj\Node\node_modules\connect\lib\middleware\router.js:183:16)
at pass (E:\proj\Node\node_modules\connect\lib\middleware\router.js:191:10)
Thanks
The two pieces of code you present are asynchronous and not synchronous!
With everyauth, to be able to handle asynchronous user creation you should use a Promise. So your code will be something like :
registerUser(function(newUserAttrs) {
var promise = this.Promise();
var login = newUserAttrs[this.loginKey()];
user.CreateNewUser(newUserAttrs.login, newUserAttrs.password, newUserAttrs.email, function(res, err) {
if(!err) {
return promise.fulfill(newUserAttrs);
}
else {
promise.fulfill(user);
}
});
})
Without promise you couldn't be sure that your new user has been added in your database. But if it doesn't matter you could have something like that:
registerUser(function(newUserAttrs) {
var login = newUserAttrs[this.loginKey()];
user.CreateNewUser(newUserAttrs.login, newUserAttrs.password, newUserAttrs.email, function(res, err) {
if (err) console.log(err);
});
return newUserAttrs;
})
Because you are doing a database query, this code has to be asynchronous. The anonymous function you pass to client.query will not be called until the database query is complete, so your callback gets called asynchronously.
You will need to treat this all as asynchronous, so for instance you'll have to trigger some other callback instead of returning the user object/throwing.