I am working with on a Lambda function that an Alexa skill can use. All i want is something simple that can read the event and send the info back to the user. For this purpose I'm using the npm library ical.js https://www.npmjs.com/package/ical with the function ical.fromURL(url, options, function(err, data) {} ) but the problem is that the function never executes. I have the following code:
var Alexa = require("alexa-sdk");
var ical = require("ical");
var test = "This is a simple test 1";
exports.handler = function(event, context) {
var alexa = Alexa.handler(event, context);
alexa.registerHandlers(handlers);
alexa.execute();
};
var handlers = {
'LaunchRequest':function() {
console.log(test);
ical.fromURL('http://lanyrd.com/topics/nodejs/nodejs.ics', {}, function(err, data) {
test = "Nothing changes";
});
console.log(test);
test.emit(':tell', 'I am done');
}
};
This is the output I get from the could watch when I do "ask simulate -l en-US -t 'start calendar read'" in ASK CLI output on cloudwatch as you can see the test text doesn't change, and would work if it was outside of the function(err, data){}. I don't believe there are any problems with reading in the calendar as the link http://lanyrd.com/topics/nodejs/nodejs.ics downloads a working ics file. The function activates if I try it in the https://npm.runkit.com/ical tool. So I am not sure what I am doing wrong. Also the skill works gives the response when tested in the alexa skill kit development.
You mistype the test.emit(':tell', 'I am done'); instead of this.emit(':tell', 'I am done');.
Also your code will not return the data from the url, because the this.emit will be return first rather than your callback function. In order to return the data, you need to put the this.emit inside the callback function of ical.fromURL.
var handlers = {
'LaunchRequest':function() {
console.log(test);
ical.fromURL('http://lanyrd.com/topics/nodejs/nodejs.ics', {}, (err, data) => {
test = "Nothing changes";
console.log(test);
// you can edit the response here base on the data you receive.
this.emit(':tell', `I am done: ${test}`);
});
}
};
In my Node-Express backend I have a route which execute certain operations and returns a status message to let the client know the response.
e.g.
server side
.post('/api/route', function(req, res) {
if (condition) { res.json({result: "ok"}); return; }
else res.json({err: "errorX"});
}
then on the client side
xhr.onreadystatechange = function() {
if (xhr.readyState == XMLHttpRequest.DONE) {
var responseJson = JSON.parse(xhr.responseText);
if (responseJson.result && result="ok") { // do something }
else if (responseJson.error && error="err1") { // do something }
else if (responseJson.error && error="err2") { // do something }
}
};
I know that in Node I can also return a statusCode
e.g.
res.status(500).send(null);
and then catch client side by
xhr.status
What is the right approach (error/result field VS status code) to handle operation's outcome in Node?
In my sort-of informed opinion, that every 'good' code, that is, one that isn't an error (such as 100-300) should just return the status code. If something is broken, it should return the status code + the error, so that debugging can go easier.
But again, this is just my opinion. Please take it with a grain of salt.
Ok, lets say I have two Models. Contract and CommLog. Both work find independently but I need many CommLog to relate to each Contract.
In the ContractSchema trying async
ContractSchema.methods.getCommLog = function getCommLog(){
var log = false;
async.parallel([
function(){
CommLog.find({commType:'contract',parent:this._id},function(err,comms){
log = comms;
});
}],
function(){return log;});
};
Where I am trying to use it
router.get('/:code', function(req, res, next) {
Contract.findOne({accessCode:req.params.code},function(err,contract){
if(err)
res.send(err);
var data;
if(contract != null){
var comms = contract.getCommLog();
data = {error:false,data:contract,commlog:comms}
}else{
data = {error:true,message:"No Contract"}
}
res.json(data);
});
});
Where it shows var comms = contract.getCommLog(); It is never returning anything because the getCommLog() is not executing async...
I think its my misunderstanding of mongoose querying, so if you understand what I am trying to accomplish, please let me know what I am doing wrong. I have tried without async which would always return false.
The find call can return all matching results with one query, so I don't think you need async here. The reason it is not populating correctly when you call res.json(data) is because you are not waiting for the method call to finish before you fire off your server response. You would be better off nesting an additional CommLogs.find call within the Contract.find call, and only sending your response once that finishes.
//pseudo code:
Contract.find({}, function(err, contract) {
if(err || !contract) {
//return error response
}
else {
CommLogs.find({contract: contract._id}, function(err, commlogs) {
if(err || !commlogs) {
//return error response 2
}
else {
res.json({errors: false, contract: contract, commlogs: commlogs});
}
});
}
}
In below code am I in callbackhell? How to overcome such scenario without using any async modules in pure javascript?
emailCallBack(e_data, email);
if (email_list.length) {
checkEmail(email_list.pop());
} else {
completionCallback();
}
The above code is copied in multiple location to make code work as expected.
function processInviteEmails(email_list, user_id, emailCallBack, completionCallback){
function checkEmail(email){
try {
check(email).isEmail();
//is valid email
checkConnected(email, user_id, function(connect_status, user_row, user_meta_row, connect_row){
var e_data;
//insert to connect and send msg to queue
if(connect_status === 'not connected'){
var cur_date = moment().format('YYYY-MM-DD');
var dbData = {
"first_name": '',
"last_name": '',
"email": email,
"user_id": user_id,
"status": "invited",
"unsubscribe_token": crypto.randomBytes(6).toString('base64'),
"created": cur_date,
"modified": cur_date
};
ConnectModel.insert(dbData, function(result){
if (result.insertId > 0) {
//send to email queue
//Queue Email
MailTemplateModel.getTemplateData('invitation', function(res_data){
if(res_data.status === 'success'){
var unsubscribe_hash = crypto.createHash("md5")
.update(dbData.unsubscribe_token + email)
.digest('hex');
var unsubscribe_link = app.locals.SITE_URL+'/unsubscribe/' + result.insertId + '/' + unsubscribe_hash;
var template_row = res_data.template_row;
var user_full_name = user_row.user_firstname+' '+ user_row.user_lastname;
var invitation_link = 'http://'+user_row.url_alias+'.'+ app.locals.SITE_DOMAIN;
var mailOptions = {
"type": 'invitation',
"to": dbData.email,
"from_name" : user_full_name,
"subject": template_row.message_subject
.replace('[[USER]]', user_full_name),
"text": template_row.message_text_body
.replace('[[USER]]', user_full_name)
.replace('[[INVITATION_LINK]]', invitation_link)
.replace('[[UNSUBSCRIBE_LINK]]', unsubscribe_link),
"html": template_row.message_body
.replace('[[USER]]', user_full_name)
.replace('[[INVITATION_LINK]]', invitation_link)
.replace('[[UNSUBSCRIBE_LINK]]', unsubscribe_link)
};
mailOptions = JSON.stringify(mailOptions);
//send email to queue
sqsHelper.addToQueue(cfg.sqs_invitation_url, mailOptions, function(data){
if(data){
e_data = null;
}
else{
e_data = new Error('Unable to Queue ');
}
emailCallBack(e_data, email);
if (email_list.length) {
checkEmail(email_list.pop());
} else {
completionCallback();
}
});
}
else{
e_data = new Error('Unable to get email template');
emailCallBack(e_data, email);
if (email_list.length) {
checkEmail(email_list.pop());
} else {
completionCallback();
}
}
});
}
else{
e_data = new Error('Unable to Insert connect');
emailCallBack(e_data, email);
if (email_list.length) {
checkEmail(email_list.pop());
} else {
completionCallback();
}
}
});
}
else{
e_data = new Error('Already connected');
emailCallBack(e_data, email);
if (email_list.length) {
checkEmail(email_list.pop());
} else {
completionCallback();
}
}
});
} catch (e) {
//invalid email
emailCallBack(e, email);
if (email_list.length) {
checkEmail(email_list.pop());
} else {
completionCallback();
}
}
}
checkEmail(email_list.pop());
}
Yes you are in callback hell. The solution assuming you don't want to use async (which I doubt you can justify other than prejudice) consists of:
1) Make more top-level functions. Each function should perform either 1 or 2 IO operations as a rule of thumb.
2) Call those functions, making your code follow a pattern of a long list of short core functions organized into business logic by a small list of control flow "glue" functions.
Instead of:
saveDb1 //lots of code
saveDb2 //lots of code
sendEmail //lots of code
Aim for:
function saveDb1(arg1, arg2, callback) {//top-level code}
function saveDb2(arg1, arg2, callback) {//top-level code}
function sendEmail(arg1, arg2, callback) {//top-level code}
function businessLogic(){//uses the above to get the work done}
3) Use more function arguments instead of relying so much on closures
4) Emit events and DECOUPLE YOUR CODE! See how you have nested code writing stuff to the database and then building an email and adding it to a queue? Don't you see how those two do not need to exist one on top of the other? Emails lend themselves very well to a core business logic emitting events and an email module listening to those events and queueing the mail.
5) Decouple application-level service connection code from specific transaction business logic. Dealing with connections to network services should be handled more broadly and not embedded with a specific set of business logic.
6) Read other modules for examples
As to should you use an async library, you can and should make up your own mind about that but AFTER you know, and know pretty well, each and every one of these approaches:
callbacks and basic functional javascript techniques
events
promises
Helper libraries (async, step, nimble, etc)
Any serious node.js developer knows how to use and work within ALL of those paradigms. Yes, everyone has their favored approach and maybe some nerd rage about the non-favored approaches, but none of these are difficult and it's bad to get set in your decision without being able to point to some non-trivial code you wrote from scratch in each paradigm. Also, you should try several helper libraries and understand how they work and why they are going to save you boilerplate. Studying the work of Tim Caswell's Step or Caolan McMahon's async is going to be very enlightening. Have you seen the everyauth source code's use of promises? I don't like it personally but I surely have to admit that the author has squeezed damn near every last bit of repetition out of that library, and the way he uses promises will turn your brain into a pretzel. These people are wizards with much to teach. Don't scoff at those libraries just for hipster points or whatever.
Also a good external resource is callbackhell.com.
"If you try to code bussiness db login using pure node.js, you go straight to callback hell"
I've recently created a simple abstraction named WaitFor to call async functions in sync mode (based on Fibers): https://github.com/luciotato/waitfor
check the database example:
Database example (pseudocode)
pure node.js (mild callback hell):
var db = require("some-db-abstraction");
function handleWithdrawal(req,res){
try {
var amount=req.param("amount");
db.select("* from sessions where session_id=?",req.param("session_id"),function(err,sessiondata) {
if (err) throw err;
db.select("* from accounts where user_id=?",sessiondata.user_ID),function(err,accountdata) {
if (err) throw err;
if (accountdata.balance < amount) throw new Error('insufficient funds');
db.execute("withdrawal(?,?),accountdata.ID,req.param("amount"), function(err,data) {
if (err) throw err;
res.write("withdrawal OK, amount: "+ req.param("amount"));
db.select("balance from accounts where account_id=?", accountdata.ID,function(err,balance) {
if (err) throw err;
res.end("your current balance is " + balance.amount);
});
});
});
});
}
catch(err) {
res.end("Withdrawal error: " + err.message);
}
Note: The above code, although it looks like it will catch the exceptions, it will not.
Catching exceptions with callback hell adds a lot of pain, and i'm not sure if you will have the 'res' parameter
to respond to the user. If somebody like to fix this example... be my guest.
using wait.for:
var db = require("some-db-abstraction"), wait=require('wait.for');
function handleWithdrawal(req,res){
try {
var amount=req.param("amount");
sessiondata = wait.forMethod(db,"select","* from session where session_id=?",req.param("session_id"));
accountdata= wait.forMethod(db,"select","* from accounts where user_id=?",sessiondata.user_ID);
if (accountdata.balance < amount) throw new Error('insufficient funds');
wait.forMethod(db,"execute","withdrawal(?,?)",accountdata.ID,req.param("amount"));
res.write("withdrawal OK, amount: "+ req.param("amount"));
balance=wait.forMethod(db,"select","balance from accounts where account_id=?", accountdata.ID);
res.end("your current balance is " + balance.amount);
}
catch(err) {
res.end("Withdrawal error: " + err.message);
}
Note: Exceptions will be catched as expected.
db methods (db.select, db.execute) will be called with this=db
Your Code
In order to use wait.for, you'll have to STANDARDIZE YOUR CALLBACKS to function(err,data)
If you STANDARDIZE YOUR CALLBACKS, your code might look like:
//run in a Fiber
function processInviteEmails(email_list, user_id, emailCallBack, completionCallback){
while (email_list.length) {
var email = email_list.pop();
try {
check(email).isEmail(); //is valid email or throw
var connected_data = wait.for(checkConnected,email,user_id);
if(connected_data.connect_status !== 'not connected') throw new Error('Already connected');
//insert to connect and send msg to queue
var cur_date = moment().format('YYYY-MM-DD');
var dbData = {
"first_name": '',
"last_name": '',
"email": email,
"user_id": user_id,
"status": "invited",
"unsubscribe_token": crypto.randomBytes(6).toString('base64'),
"created": cur_date,
"modified": cur_date
};
result = wait.forMethod(ConnectModel,'insert',dbData);
// ConnectModel.insert shuold have a fn(err,data) as callback, and return something in err if (data.insertId <= 0)
//send to email queue
//Queue Email
res_data = wait.forMethod(MailTemplateModel,'getTemplateData','invitation');
// MailTemplateModel.getTemplateData shuold have a fn(err,data) as callback
// inside getTemplateData, callback with err=new Error('Unable to get email template') if (data.status !== 'success')
var unsubscribe_hash = crypto.createHash("md5")
.update(dbData.unsubscribe_token + email)
.digest('hex');
var unsubscribe_link = app.locals.SITE_URL+'/unsubscribe/' + result.insertId + '/' + unsubscribe_hash;
var template_row = res_data.template_row;
var user_full_name = user_row.user_firstname+' '+ user_row.user_lastname;
var invitation_link = 'http://'+user_row.url_alias+'.'+ app.locals.SITE_DOMAIN;
var mailOptions = {
"type": 'invitation',
"to": dbData.email,
"from_name" : user_full_name,
"subject": template_row.message_subject
.replace('[[USER]]', user_full_name),
"text": template_row.message_text_body
.replace('[[USER]]', user_full_name)
.replace('[[INVITATION_LINK]]', invitation_link)
.replace('[[UNSUBSCRIBE_LINK]]', unsubscribe_link),
"html": template_row.message_body
.replace('[[USER]]', user_full_name)
.replace('[[INVITATION_LINK]]', invitation_link)
.replace('[[UNSUBSCRIBE_LINK]]', unsubscribe_link)
};
mailOptions = JSON.stringify(mailOptions);
//send email to queue ... callback(err,data)
wait.forMethod(sqsHelper,'addToQueue',cfg.sqs_invitation_url, mailOptions);
} catch (e) {
// one of the callback returned err!==null
emailCallBack(e, email);
}
} // loop while length>0
completionCallback();
}
// run the loop in a Fiber (keep node spinning)
wait.launchFiber(processInviteEmails,email_list, user_id, emailCallBack, completionCallback);
see? no callback hell
I've put another solution in my blog. It is ugly but it is the most readable thing I could do with pure javascript.
var flow1 = new Flow1(
{
execute_next_step: function(err) {
if (err) {
console.log(err);
};
}
}
);
flow1.execute_next_step();
function Flow1(parent_flow) {
this.execute_next_step = function(err) {
if (err) return parent_flow.execute_next_step(err);
if (!this.next_step) this.next_step = 'START';
console.log('Flow1:', this.next_step);
switch (this.next_step) {
case 'START':
this.next_step = 'FIRST_ASYNC_TASK_FINISHED';
firstAsyncTask(this.execute_next_step.bind(this));
break;
case 'FIRST_ASYNC_TASK_FINISHED':
this.firstAsyncTaskReturn = arguments[1];
this.next_step = 'ANOTHER_FLOW_FINISHED';
this.another_flow = new AnotherFlow(this);
this.another_flow.execute_next_step();
break;
case 'ANOTHER_FLOW_FINISHED':
this.another_flow_return = arguments[1];
this.next_step = 'FINISH';
this.execute_next_step();
break;
case 'FINISH':
parent_flow.execute_next_step();
break;
}
}
}
function AnotherFlow(parent_flow) {
this.execute_next_step = function(err) {
if (err) return parent_flow.execute_next_step(err);
if (!this.next_step) this.next_step = 'START';
console.log('AnotherFlow:', this.next_step);
switch (this.next_step) {
case 'START':
console.log('I dont want to do anything!. Calling parent');
parent_flow.execute_next_step();
break;
}
}
}
I've been working with Meteor and the stripe package to try and make a customer. So First I have my client side code which calls a method on the server so when clicked i have in the client.js:
Meteor.call('usersignup', function (error, result) {
console.log (result);
});
So this calls the Method on the server.js:
var Future = Npm.require('fibers/future');
var stripe = StripeAPI('my key'); // secret stripe API key
Meteor.methods({
usersignup: function(cusEmail){
var fut = new Future();
stripe.customers.create(
{ email: cusEmail },
function(err, customer) {
if (err) {
console.log(err);
fut.ret;
}
fut.ret(customer);
}
);
return fut.wait();
},
userfail: function(cusid){
var fut = new Future();
stripe.customers.retrieve(cusid, function(err, result) {
if(err){
console.log(err);
fut.ret;
}
fut.ret(err, result);
});
return fut.wait();
}
});
Now this works and creates a customer when I log onto the stripe.com dashboard but I'm trying to get the response returned to the client well at least the customer id for now and print it in the console. This is where I can't seem to get it to work. It'll log undefined when I do console.log(result). Any ideas?
EDIT: So I put the fiber and the stripe key as global variables now and don't get an error but the returns don't seem to be returning any values. so on the client side I have:
'click #signupsubmit': function (event) {
console.log("hello");
var whatis = getVal(); // function gets value of forms and returns object
var testid;
var cusid = Meteor.call('usersignup', whatis.email, function (error, result) {
if (error) {
console.log(err.message);
return;
}
console.log(result);
console.log("meteor call");
testid = result;
return (result);
});
console.log("outside call");
console.log(testid);
console.log(cusid);
},
});
So i've been running some console.log tests and it seems it executes the meteor.call and keeps going down the line. Console.log of both testid and cusid return undefined but a couple seconds later I receive the console.log of result and the string "meteor call" from inside the meteor.call. Is there a way to wait for the meteor call to finish then run the rest of what is in my click function? so console output will go like:
"hello"
"outside call"
test id undefined
cusid undefined
"meteor call"
"result"
Keep in mind that the stripe API doesn't use Fibers. You need to put it in manually. The callback doesn't reach the client because by then it would have already got a response (its async)
You can use something like this to wait for a result from the stripe callback before a result is returned to the client:
var stripe = StripeAPI('mykeygoeshere'); // secret stripe API key
var Future = Npm.require('fibers/future');
var fut = new Future();
stripe.customers.create(
{ email: 'hello#example.org' },
function(err, customer) {
if (err) {
console.log(err.message);
fut.ret;
}
fut.ret("customer id", customer.id);
}
);
return fut.wait();
Here a Future is used and it waits for a result to be received from the stripe callback before a result is returned to the client.
More info can be found on Fibers/Futures & Synchronous Callbacks incuding how to go about them & when to use them:
Meteor: Calling an asynchronous function inside a Meteor.method and returning the result
https://github.com/laverdet/node-fibers
https://gist.github.com/possibilities/3443021
Here's something simpler. Meteor now has Meteor.wrapAsync() for this kind of situation:
var stripe = StripeAPI("key");
Meteor.methods({
yourMethod: function(callArg) {
var charge = Meteor.wrapAsync(stripe.charges.create, stripe.charges);
charge({
amount: amount,
currency: "usd",
//I passed the stripe token in callArg
card: callArg.stripeToken,
}, function(err, charge) {
if (err && err.type === 'StripeCardError') {
// The card has been declined
throw new Meteor.Error("stripe-charge-error", err.message);
}
//Insert your 'on success' code here
});
}
});
I found this post really helpful:
Meteor: Proper use of Meteor.wrapAsync on server