I am looking into solutions to call some C# code from within my NodeJS application. I came across EdgeJS, which seems to make this possible. However, I came across part of their code that is confusing to me, because I see an async keyword without an accompanying await. And while I'm more familiar with JS than C#, my understanding is that in BOTH you need to include an await with any async. This is the code in question, where some multi-line C# code is defined within backticks:
var edge = require('edge');
var helloWorld = edge.func(`
async (input) => {
return ".NET Welcomes " + input.ToString();
}
`);
helloWorld('JavaScript', function (error, result) {
if (error) throw error;
console.log(result);
});
Can someone explain how this is working, considering await doesn't appear anywhere?
Related
Currently I'm working on a project in node js. Specifically I'm using soem boiler plate for adobe CEP which allows you to run some js in a panel in their programs. In the code there is the following code that works fine.
mysqlConn.query(query, function(err, result) {
do something with error and result})
When this is execute it gives me an error or result depending on if there is data or there was a problem etc. What I need to do is to run another function after this executes and gives me the result. My knowledge of promises is limited (even thou I've read extensively on it and done tons of tutorials). In my limited knowledge I assume mysqlConn.query returns a promise. So I was assuming I can just do this:
mysqlConn.query(query, function(err, result) {
do something with error and result})
.then(console.log('anything here?'));
This logs to the console 'anything here?' but it also gives me this error in the console.
Uncaught TypeError: mysqlConn.query(...).then is not a function
Any idea what I'm doing wrong or how I can achieve the desired results?
This indicates the mysqlConn.query method does not return a Promise
Instead, you will need to "promisify" the method so it can be changed with .then() and friends:
const myFunc = new Promise((resolve, reject) => {
return mysqlConn.query(query, function(err, result) {
if (err) reject(err);
return resolve(result);
});
});
Now we have myFunc, a Promise-based interface wrapping the callback-basked query function. We can use it like so:
return myFunc()
.then((result) => { ... }) // result will be the result of the query
.catch((err) => { .. } ) // err from the query as well
This can also be achieved in a slightly more involved way through other tools, but I highly recommend you understand this example first.
Hi I'm a newbie in nodejs as far as I'm concerned nodejs is event-driven which is a powerful feature in it.
I have been learning nodejs from last few days and try to build restful apis in it with mongodb, but I'm not able to use its event-driven architecture in apis below is my sudo code
//routes
app.get('/someUrl', SomeClass.executeSomeController);
//controller
class SomeClass {
async executeSomeController(req, res){
let response = awaitSomeHelper.executeQueryAndBusinessLogic(req.body);
res.send(response)
}
}
As per my understanding I have written normal code as I used to write using Ror or PHP The only difference I found that the controller is running asynchronous which does not happens in Ror or Php.
How can I use event-driven architecture to build restful apis
Hope I can cover your question. Basically in some cases 'event-driven architecture' term can be explained differently. In one case it's a basic core NodeJS flow that explains all the async functions. In another case, the root of the question can be related to events, event emitter etc.
But the main idea that you have to wait for all the asynchronous actions. In order to avoid thread blocking it goes further and handles the rest of your code without waiting for heavy requests. And there we have to know how to handle this async functionality.
Basic Async Flow
As I understand, you've got questions related to async operations in NodeJS. That's a root of the technology - all the heavy operations will be handled asynchronously. It's all about V8 and Event Loop.
So in order to work with asynchronous operations, you may use callback functions, promises or async-await syntax.
Callback Functions
function asyncFunction(params, callback) {
//do async stuff
callback(err, result);
}
function callbackFunction(err, result) {
}
asyncFunction(params, callbackFunction);
Promises
promiseFunction()
.then(anotherPromiseFunction)
.then((result) => {
//handle result
})
.catch((err) => {
//handle error
});
async-await
function anotherAsyncFunction() {
//do async stuff
}
const asycnFunction = async (params) => {
const result = await anotherAsyncFunction();
return result;
};
Events/Event Emitter
const fs = require('fs');
const filePath = './path/to/your/file';
const stream = fs.createReadStream(filePath);
stream.on('data', (data) => {
//do something
});
stream.on('end', () => {
//do something;
});
stream.on('error', (err) => {
//do something;
});
You may use these methods depends on the situation and your needs. I recommend skipping callback functions as we have modern ways to work in async flow (promises and async-await). By the way, 'async-await' returns promises as well.
Here is the example of a simple Express JS Server (pretty old syntax), but still valid. Please feel free to check and write questions:
https://github.com/roman-sachenko/express-entity-based
Here is a list of articles I'd recommend you:
https://blog.risingstack.com/node-js-at-scale-understanding-node-js-event-loop/
https://blog.risingstack.com/mastering-async-await-in-nodejs/
I am new to nodejs/expressjs and mongodb. I am trying to create an API that exposes data to my mobile app that I am trying to build using Ionic framework.
I have a route setup like this
router.get('/api/jobs', (req, res) => {
JobModel.getAllJobsAsync().then((jobs) => res.json(jobs)); //IS THIS THe CORRECT WAY?
});
I have a function in my model that reads data from Mongodb. I am using the Bluebird promise library to convert my model functions to return promises.
const JobModel = Promise.promisifyAll(require('../models/Job'));
My function in the model
static getAllJobs(cb) {
MongoClient.connectAsync(utils.getConnectionString()).then((db) => {
const jobs = db.collection('jobs');
jobs.find().toArray((err, jobs) => {
if(err) {
return cb(err);
}
return cb(null, jobs);
});
});
}
The promisifyAll(myModule) converts this function to return a promise.
What I am not sure is,
If this is the correct approach for returning data to the route callback function from my model?
Is this efficient?
Using promisifyAll is slow? Since it loops through all functions in the module and creates a copy of the function with Async as suffix that now returns a promise. When does it actually run? This is a more generic question related to node require statements. See next point.
When do all require statements run? When I start the nodejs server? Or when I make a call to the api?
Your basic structure is more-or-less correct, although your use of Promise.promisifyAll seems awkward to me. The basic issue for me (and it's not really a problem - your code looks like it will work) is that you're mixing and matching promise-based and callback-based asynchronous code. Which, as I said, should still work, but I would prefer to stick to one as much as possible.
If your model class is your code (and not some library written by someone else), you could easily rewrite it to use promises directly, instead of writing it for callbacks and then using Promise.promisifyAll to wrap it.
Here's how I would approach the getAllJobs method:
static getAllJobs() {
// connect to the Mongo server
return MongoClient.connectAsync(utils.getConnectionString())
// ...then do something with the collection
.then((db) => {
// get the collection of jobs
const jobs = db.collection('jobs');
// I'm not that familiar with Mongo - I'm going to assume that
// the call to `jobs.find().toArray()` is asynchronous and only
// available in the "callback flavored" form.
// returning a new Promise here (in the `then` block) allows you
// to add the results of the asynchronous call to the chain of
// `then` handlers. The promise will be resolved (or rejected)
// when the results of the `job().find().toArray()` method are
// known
return new Promise((resolve, reject) => {
jobs.find().toArray((err, jobs) => {
if(err) {
reject(err);
}
resolve(jobs);
});
});
});
}
This version of getAllJobs returns a promise which you can chain then and catch handlers to. For example:
JobModel.getAllJobs()
.then((jobs) => {
// this is the object passed into the `resolve` call in the callback
// above. Do something interesting with it, like
res.json(jobs);
})
.catch((err) => {
// this is the error passed into the call to `reject` above
});
Admittedly, this is very similar to the code you have above. The only difference is that I dispensed with the use of Promise.promisifyAll - if you're writing the code yourself & you want to use promises, then do it yourself.
One important note: it's a good idea to include a catch handler. If you don't, your error will be swallowed up and disappear, and you'll be left wondering why your code is not working. Even if you don't think you'll need it, just write a catch handler that dumps it to console.log. You'll be glad you did!
I'm coming from a java background so a bit of a newbie on Javascript conventions needed for Lambda.
I've got a lambda function which is meant to do several AWS tasks in a particular order, depending on the result of the previous task.
Given that each task reports its results asynchronously, I'm wondering if the right way make sure they all happen in the right sequence, and the results of one operation are available to the invocation of the next function.
It seems like I have to invoike each function in the callback of the prior function, but seems like that will some kind of deep nesting and wondering if that is the proper way to do this.
For example on of these functions requires a DynamoDB getItem, following by a call to SNS to get an endpoint, followed by a SNS call to send a message, followed by a DynamoDB write.
What's the right way to do that in lambda javascript, accounting for all that asynchronicity?
I like the answer from #jonathanbaraldi but I think it would be better if you manage control flow with Promises. The Q library has some convenience functions like nbind which help convert node style callback API's like the aws-sdk into promises.
So in this example I'll send an email, and then as soon as the email response comes back I'll send a second email. This is essentially what was asked, calling multiple services in sequence. I'm using the then method of promises to manage that in a vertically readable way. Also using catch to handle errors. I think it reads much better just simply nesting callback functions.
var Q = require('q');
var AWS = require('aws-sdk');
AWS.config.credentials = { "accessKeyId": "AAAA","secretAccessKey": "BBBB"};
AWS.config.region = 'us-east-1';
// Use a promised version of sendEmail
var ses = new AWS.SES({apiVersion: '2010-12-01'});
var sendEmail = Q.nbind(ses.sendEmail, ses);
exports.handler = function(event, context) {
console.log(event.nome);
console.log(event.email);
console.log(event.mensagem);
var nome = event.nome;
var email = event.email;
var mensagem = event.mensagem;
var to = ['email#company.com.br'];
var from = 'site#company.com.br';
// Send email
mensagem = ""+nome+"||"+email+"||"+mensagem+"";
console.log(mensagem);
var params = {
Source: from,
Destination: { ToAddresses: to },
Message: {
Subject: {
Data: 'Form contact our Site'
},
Body: {
Text: {
Data: mensagem,
}
}
};
// Here is the white-meat of the program right here.
sendEmail(params)
.then(sendAnotherEmail)
.then(success)
.catch(logErrors);
function sendAnotherEmail(data) {
console.log("FIRST EMAIL SENT="+data);
// send a second one.
return sendEmail(params);
}
function logErrors(err) {
console.log("ERROR="+err, err.stack);
context.done();
}
function success(data) {
console.log("SECOND EMAIL SENT="+data);
context.done();
}
}
Short answer:
Use Async / Await — and Call the AWS service (SNS for example) with a .promise() extension to tell aws-sdk to use the promise-ified version of that service function instead of the call back based version.
Since you want to execute them in a specific order you can use Async / Await assuming that the parent function you are calling them from is itself async.
For example:
let snsResult = await sns.publish({
Message: snsPayload,
MessageStructure: 'json',
TargetArn: endPointArn
}, async function (err, data) {
if (err) {
console.log("SNS Push Failed:");
console.log(err.stack);
return;
}
console.log('SNS push suceeded: ' + data);
return data;
}).promise();
The important part is the .promise() on the end there. Full docs on using aws-sdk in an async / promise based manner can be found here: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/using-promises.html
In order to run another aws-sdk task you would similarly add await and the .promise() extension to that function (assuming that is available).
For anyone who runs into this thread and is actually looking to simply push promises to an array and wait for that WHOLE array to finish (without regard to which promise executes first) I ended up with something like this:
let snsPromises = [] // declare array to hold promises
let snsResult = await sns.publish({
Message: snsPayload,
MessageStructure: 'json',
TargetArn: endPointArn
}, async function (err, data) {
if (err) {
console.log("Search Push Failed:");
console.log(err.stack);
return;
}
console.log('Search push suceeded: ' + data);
return data;
}).promise();
snsPromises.push(snsResult)
await Promise.all(snsPromises)
Hope that helps someone that randomly stumbles on this via google like I did!
I don't know Lambda but you should look into the node async library as a way to sequence asynchronous functions.
async has made my life a lot easier and my code much more orderly without the deep nesting issue you mentioned in your question.
Typical async code might look like:
async.waterfall([
function doTheFirstThing(callback) {
db.somecollection.find({}).toArray(callback);
},
function useresult(dbFindResult, callback) {
do some other stuff (could be synch or async)
etc etc etc
callback(null);
],
function (err) {
//this last function runs anytime any callback has an error, or if no error
// then when the last function in the array above invokes callback.
if (err) { sendForTheCodeDoctor(); }
});
Have a look at the async doco at the link above. There are many useful functions for serial, parallel, waterfall, and many more. Async is actively maintained and seems very reliable.
good luck!
A very specific solution that comes to mind is cascading Lambda calls. For example, you could write:
A Lambda function gets something from DynamoDB, then invokes…
…a Lambda function that calls SNS to get an endpoint, then invokes…
…a Lambda function that sends a message through SNS, then invokes…
…a Lambda function that writes to DynamoDB
All of those functions take the output from the previous function as input. This is of course very fine-grained, and you might decide to group certain calls. Doing it this way avoids callback hell in your JS code at least.
(As a side note, I'm not sure how well DynamoDB integrates with Lambda. AWS might emit change events for records that can then be processed through Lambda.)
Just saw this old thread. Note that future versions of JS will improve that. Take a look at the ES2017 async/await syntax that streamlines an async nested callback mess into a clean sync like code.
Now there are some polyfills that can provide you this functionality based on ES2016 syntax.
As a last FYI - AWS Lambda now supports .Net Core which provides this clean async syntax out of the box.
I would like to offer the following solution, which simply creates a nested function structure.
// start with the last action
var next = function() { context.succeed(); };
// for every new function, pass it the old one
next = (function(param1, param2, next) {
return function() { serviceCall(param1, param2, next); };
})("x", "y", next);
What this does is to copy all of the variables for the function call you want to make, then nests them inside the previous call. You'll want to schedule your events backwards. This is really just the same as making a pyramid of callbacks, but works when you don't know ahead of time the structure or quantity of function calls. You have to wrap the function in a closure so that the correct value is copied over.
In this way I am able to sequence AWS service calls such that they go 1-2-3 and end with closing the context. Presumably you could also structure it as a stack instead of this pseudo-recursion.
I found this article which seems to have the answer in native javascript.
Five patterns to help you tame asynchronis javascript.
By default Javascript is asynchronous.
So, everything that you have to do, it's not to use those libraries, you can, but there's simple ways to solve this. In this code, I sent the email, with the data that comes from the event, but if you want, you just need to add more functions inside functions.
What is important is the place where your context.done(); is going to be, he is going to end your Lambda function. You need to put him in the end of the last function.
var AWS = require('aws-sdk');
AWS.config.credentials = { "accessKeyId": "AAAA","secretAccessKey": "BBBB"};
AWS.config.region = 'us-east-1';
var ses = new AWS.SES({apiVersion: '2010-12-01'});
exports.handler = function(event, context) {
console.log(event.nome);
console.log(event.email);
console.log(event.mensagem);
nome = event.nome;
email = event.email;
mensagem = event.mensagem;
var to = ['email#company.com.br'];
var from = 'site#company.com.br';
// Send email
mensagem = ""+nome+"||"+email+"||"+mensagem+"";
console.log(mensagem);
ses.sendEmail( {
Source: from,
Destination: { ToAddresses: to },
Message: {
Subject: {
Data: 'Form contact our Site'
},
Body: {
Text: {
Data: mensagem,
}
}
}
},
function(err, data) {
if (err) {
console.log("ERROR="+err, err.stack);
context.done();
} else {
console.log("EMAIL SENT="+data);
context.done();
}
});
}
I'm learning node.js coming from a PHP background with a limited JavaScript level. I think I got over now the change of mindset implied by the asynchronous approach. And I love it.
But, as many others before me, I quickly understood the concrete meaning of the "pyramid of doom".
So I build these little 'dummy' route and view to understand how to properly use Async.js. I just spend the last 5 hours writing the following code (rewritten of course tens of times). It works, but I wonder how I could go further and made this code more simple (less verbose, easier to read and maintain).
I found many resources on the web and especially here, but always by bits of info here and there.
I'm guessing at this point that I should use "bind" and "this" with async.apply to make to shorten the 2 last functions called by the waterfall.
The issue is to get the object "db" defined so I can use the "collection" method on it (for the second function).
I really searched an example in Google, but it's surprising that you don't get straightforward examples looking for "async waterfall bind" (as well as many keyword variations I tried). There are answers of course but none seems relevant to this particular issue... ore, quite possibly, I haven't understood them.
Can someone help me on this? I'll be quite grateful.
app.get('/dummy',
function(req, res) {
var MongoClient = require('mongodb').MongoClient;
async.waterfall(
[
async.apply(MongoClient.connect, 'mongodb://localhost:27017/mybdd'),
function(db, callback) {
db.collection('myCollection', callback);
},
function(collection, callback) {
collection.find().sort({"key":-1}).limit(10).toArray(callback);
}
], function(err, results) {
if (err) console.log('Error :', err);
else { res.render('dummy.jade', { title:'dummy', results: results} ); }
}
);
}
);
If you're using the mongodb JS Driver, then this should work:
async.waterfall(
[
function (cb) {
new MongoClient(...)
.connect('mongodb://localhost:27017/mybdd', cb);
},
function (db, callback) {
db.collection('myCollection', callback);
},
...
Alternatively, if you want to use async.apply, just pass an instance of MongoClient
async.apply(new MongoClient(...).connect, 'mongodb://localhost:27017/mybdd')
I've recently created a simple abstraction named WaitFor to call async functions in sync mode (based on Fibers): https://github.com/luciotato/waitfor
I'm not familiar with mongodb client, so i'll be mostly guessing what you're trying to do:
using WaitFor your code will be:
var MongoClient = require('mongodb').MongoClient;
var wait = require('waitfor');
app.get('/dummy', function(req, res) {
// handle request in a Fiber, keep node spinning
wait.launchFiber(handleDummy,req,res)
}
);
function handleDummy(req, res) {
try {
var db = wait.for(MongoClient.connect, 'mongodb://localhost:27017/mybdd');
var collection = wait.forMethod(db,'collection','myCollection');
var results = wait.forMethod(collection.,'sort',{"key":-1}).toArray();
res.render('dummy.jade', { title:'dummy', results: results} );
}
catch(err) {
res.render('error.jade', { title:'error', message: err.message} );
}
};