How to async require in nodejs - node.js

I'm using bluebird to init various type of db connections.
// fileA.js
Promise.all(allConnectionPromises)
.then(function (theModels) {
// then i want to do module.exports here
// console.log here so the expected output
module.exports = theModels
})
So that I can require that file above from another file. however if I do that, i get {}.
let A = require('./fileA') // i get {} here
Any idea how to do this?

You can't magically make an async operation into a synchronous operation in Javascript. So, an async operation will require async coding techniques (callbacks or promises). The same is true in regular coding as in module startup/initialization.
The usual design pattern for dealing with this issue is to give your module a constructor function that you pass a callback to and when you call that constructor function, it will call the callback when the async result is done and then any further code that uses that async result must be in that callback.
Or, since you're already using promises, you can just export a promise that the calling code can use.
// fileA.js
module.exports = Promise.all(allConnectionPromises);
Then, in code that uses this:
require('./fileA').then(function(theModels) {
// access theModels here
}, function(err) {
console.log(err);
});
Note, when doing it like this, the exported promise serves as a convenient cache for theModels too since every other module that does require('./fileA') will get the same promise back and thus the same resolved value without re-executing the code behind getting the models.
Though I think the promises version is probably cleaner especially since you're already using promises within the module, here's what the constructor version would look like for comparison:
// fileA.js
var p = Promise.all(allConnectionPromises);
module.exports = function(callback) {
p.then(function(theModels) {
callback(null, theModels);
}, function(err) {
callback(err);
});
}
Then, in code that uses this:
require('./fileA')(function(err, theModels) {
if (err) {
console.log(err);
} else {
// use theModels here
}
});

Related

await inside async controller function [duplicate]

I am working on a simple TCP client for a server and am using the latest node 7.6 because of the async/await functions. I'm new to node and asynchronous coding, so I apologize if this is stupidly easy.
I want to run a function that calls the callServer() function with specific parameters, wait until it finishes getting the data, and return the data as a variable.
Here is my code:
'use strict'
const net = require('net')
var formattedJson = funcRequestToJson("Server.GetStatus", false)
doThings()
async function doThings() {
var info = await callServer()
}
async function callServer() {
var client = new net.Socket()
client.connect(1705, '192.168.28.16', () => {
console.log('connected to server')
client.write(formattedJson)
})
client.on('data', (data) => {
client.destroy()
//return await data
console.log(data.toString())
})
client.on('close', () => {
})
}
// method and paramBool are always required
// macAddress, paramKey, paramValue are required for if paramBool is true
function funcRequestToJson(method, paramBool, macAddress, paramKey, paramValue) {
var objectRequest = {}
objectRequest[ "jsonrpc" ] = "2.0"
objectRequest[ "method" ] = method
objectRequest[ "id" ] = 0
if (paramBool == true) {
objectRequest[ "params" ] = {
"client": macAddress,
[paramKey]: paramValue
}
}
var json = (JSON.stringify(objectRequest) + '\r\n')
return json
}
So I didn't declare objectRequest() as async because it's not waiting on the server, but I think callServer() should be async, right? I know this can be done with promises, but I wanted to use async/await and this seems to be right.
Now, I want to return the data that comes from inside callServer() and client.on('data', (data) but I can't seem to figure out how to do it asynchronously. I would think there'd be a way to make an async function and call it with await like I tried (await return data) but it never works right.
I'm sorry if this is terribly convoluted, but I've been poring over async node tutorials for the past week and am still stuck.
Thanks!
Async/Await relies on code that uses promises for async operations. So you need to return a promise from any async operation in order to use it with async/await.
So, callServer() needs to return a promise that is resolved when the async operation inside it is done. In fact, you can only await an async operation in a function if that function returns a promise. await saves you from having to write .then() handlers on promises, but async/await does not magically know when async operations are done. You still have to wrap them in promises.
Here's an example of how you could make callServer() return a promise:
async function callServer(formattedJson) {
return new Promise((resolve, reject) => {
let client = new net.Socket()
client.connect(1705, '192.168.28.16', () => {
console.log('connected to server')
client.write(formattedJson)
})
client.on('data', (data) => {
resolve(data);
client.destroy()
})
client.on('close', () => {
})
client.on('error', reject);
});
}
Sample Usage:
try {
var info = await callServer(funcRequestToJson("Server.GetStatus", false));
} catch(e) {
// error here
}
Just to show you what await is doing, in ES5 (without async/await), the sample usage would be this:
callServer(funcRequestToJson("Server.GetStatus", false)).then(info => {
// info is available here
}).catch(err => {
// error here
});
So, await is just letting you avoid writing the .then() handler. Internally in the execution of the code, there's just a promise that the interpreter sets up a .then() handler for so it will know when that promise is resolved. This is syntactical sugar, not really any magic with async operations. You still have to use promises for all your async operations. You can use await instead of .then() and your code will appear more sequential even though it's really the same under the covers.
People think that await allows one to write asynchronous code as if it was synchronous, but that isn't really the case, especially if you handle errors and understand concurrency issues. It will look more synchronous, but isn't actually any more synchronous than it was with .then() in ES5.
And, watch out for error handling. In people's quest to write synchronous looking code, people seem to completely forget to handle rejected promises in their async operations (which are handled with try/catch when using await). I'm personally not yet convinced that ES6 is a step forward when it comes to error handling as the early indications are that ES6 seems to encourage people to just forget about error handling or get lazy and not do it, whereas it's easier when using .then() to just know that there should be a .catch() somewhere for it to be solid code. Maybe that's just a learning process, but it seems to be an early issue when people use await.

Node.js consecutive method calls with nested callback formatting

So I'm new to Node.js and Im just wondering if the way I have my code setup makes sense. Im coming from a Java background so the nested callback structure is new. I have a Node program that runs a bunch of code that I broke down into different methods. The thing is that the methods need to be called in order. My code has this structure right now:
functionOne(data, callback(err) {
functionTwo(data, callback(err) {
functionThree(data, callback(err) {
functionFour(data, callback(err) {
//Code
});
});
});
});
This is very minimalistic, but is this structure ok? With Java, I'd take the return values of all the methods, then just pass them to the next function. From my understanding so far, the Java approach I just mentioned is one of the main things that Node.js was trying to eliminate. But anyway... Does that structure look ok, and is that how its intended to look? Just want to be sure that I'm not making any major errors with Node in general. Thanks!
Your code structure looks fine if you work with callback pattern.
But if you're interested in make your code cleaner and readable you would like to use Promises in your asynchronous function, so instead of pass a callback to your functions you could do something like this :
function asyncFunction (data){
return new Promise(function(resolve, reject){
// Do something with data
// Here you can call reject(error) to throw an error
resolve();
});
}
And instead of nested function callbacks you can call then method of Promise.
asyncFunction(data)
.then(function(){
// Promise resolved
// Something has been done with data
});
With Promises you can also execute async fuctions in parallel :
Promise.all([asyncFunctionA(data), asyncFunctionB(data), asyncFunctionC(data)])
.then(function(){...});
EDIT
If you need to pass values of one function to another, your code should look like this :
asyncFunctionA(data)
.then(function(dataA){
return asyncFunctionB(dataA);
})
.then(function(dataB){
return asyncFunctionC(dataB);
})
.then(function(dataC){
// ...
});
You should try to use promises to avoid your callback hell, so it could be something like these...
const Q = require('q'); // you can do a research for this module.
var myModule = {};
myModule.functionOne = (params) => {
const deferred = Q.defer(); // wait for this to complete
// body function
deferred.resolve(data); // this would be the result of this function
return deferred.promise; // data is the output on your function
}
myModule.functionTwo = (params) => {
const deferred = Q.defer(); // wait for this to complete
// body function
deferred.resolve(data); // this would be the result of this function
return deferred.promise; // data is the output on your function
}
myModule.doAll = (params) => {
myModule.functionOne(params)
.then((outputFunctionOne) => {
// this is called after functionOne ends
return myModule.functionTwo(outputFunctionOne);
})
.then((outputFunctionTwo) => {
// this is called after function 2 ends
if (outputFunctionTwo.success) {
// if everything ok, resolve the promise with the final output
deferred.resolve(outputFunctionTwo);
} else {
// reject the promise with an error message
deferred.reject('error');
}
})
.fail((err) => {
// this is call if the promise is rejected or an exception is thrown
console.log(err); // TODO: Error handling
})
.done();
}
module.exports = myModule;
You can Chain as many promises as you want really easily, that way you get rid of the callback hell. Best part, you can do promises on Javascript or Node.js
Reference Link https://github.com/kriskowal/q
Hope this helps
Most of the other answers give Promise/A as the answer to your callback woes. This is correct, and will work for you. However I'd like to give you another option, if you are willing to drop javascript as your working language.
Introducing Iced Coffee, a branch of the CoffeeScript project.
With Iced Coffee you would write:
await functionOne data, defer err
await functionTwo data, defer err2
await functionThree data, defer err3
//etc
This then compiles to the CoffeeScript:
functionOne data, (err) ->
functionTwo data, (err2) ->
functionThree data, (err3) ->
//etc
Which then compiles to your Javascript.
functionOne(data, callback(err) {
functionTwo(data, callback(err2) {
functionThree(data, callback(err3) {
//etc
});
});
});

What is cb() in Node?

Where are people getting cb() from, is this a Node thing or vanilla JS thing?
For example:
Managing Node.js Callback Hell with Promises, Generators and Other Approaches
they're using cb() to I guess callback and return an error or a value or both in some cases depending on what the callback function sig is?
cb in the context you're describing it is how a vanilla callback function is passed into a (typically) asynchronous function, which is a common pattern in node.js (it's sometimes labelled next, but you can call it bananas if you so desire - it's just an argument).
Typically the first argument is an error object (often false - if all went as planned) and subsequent arguments are data of some form.
For example:
function myAsyncFunction(arg1, arg2, cb) {
// async things
cb(false, { data: 123 });
}
then using this function:
myAsyncFunction(10, 99, function onComplete(error, data) {
if (!error) {
// hooray, everything went as planned
} else {
// disaster - retry / respond with an error etc
}
});
Promises are an alternative to this design pattern where you would return a Promise object from myAsyncFunction
For example:
function myAsyncFunction2(arg1, arg2) {
return new Promise(function resolution(resolve, reject, {
// async things
resolve({ data: 123 });
});
}
then using this function:
myAsyncFunction2(10, 99)
.then(function onSuccess(data) {
// success - send a 200 code etc
})
.catch(function onError(error) {
// oh noes - 500
});
They're basically the same thing, just written slightly differently. Promises aren't supported especially widely in a native form, but if put through a transpiler (I'd recommend babel) during a build step they should perform reliably enough in a browser too.
Callbacks will always work in a browser with no shimming / transpilation.
node.js has lots of asynchronous operations that take a completion callback as an argument. This is very common in various node.js APIs.
The node.js convention for this callback is that the first argument passed to the callback is an error code. A falsey value for this first argument means that there is no error.
For example:
fs.readFile("test.txt", function(err, data) {
if (!err) {
console.log("file data is: " + data);
}
});
A function you create yourself may also define it's own callback in order to communicate the end of one or more asynchronous operations.
function getData(id, cb) {
var fname = "datafile-" + id + ".txt";
fs.readFile(fname, function(err, data) {
if (err) {
cb(err);
} else if (data.slice(0, 6) !== "Header"){
// proper header not found at beginning of file data
cb(new Error("Invalid header"));
} else {
cb(0, data);
}
});
}
// usage:
getData(29, function(err, data) {
if (!err) {
console.log(data);
}
});
From the Vanilla JS, you can declare a function and pass throuw parameters a declaration of another function, that can called async
https://developer.mozilla.org/en-US/docs/Glossary/Callback_function

How do I sinon.stub a nested method with a callback?

I need to test a method that includes a sub-method which makes a call to an API server. I’d like to stud this internal sub-method, but I can’t seem to do that. Here’s an example:
var requests = require('./requests.js');
var utilityClass = {
methodCreatesObject: function (callback) {
// Here’s the method I’m trying to stub:
requests.makeCallToAPI(requestObject, function (err, responseFromAPI) {
doSomethingWithResponse(responseFromAPI, function (err, finalObject) {
if (err) {
callback(err, null);
} else {
callback(null, finalObject); // <- Want to test the value of finalObject
}
});
});
}
}
So, my test looks something like this (updated to show loading requests.js before utility.js):
var should = require('should'),
Joi = require('joi'),
sinon = require('sinon'),
requests = require('../lib/modules/requests.js'),
utility = require('../lib/modules/utility.js')
;
// Start my tests:
describe('Method', function () {
before(function () {
var fakeAPIresponse = { ... }
sinon.stub(requests, 'makeCallToAPI').yield(null, fakeAPIresponse);
});
it('should produce a well-formed finalObject', function (done) {
utilityClass.methodCreatesObject(function (err, response) {
if (err) {
done(err);
} else {
response.should.do.this.or.that;
done();
}
});
});
});
As I understand it, .yields() should try to run the first callback it detects in the arguments and feed its own arguments to it (resulting in doSomethingWithResponse(responseFromAPI, function () {...})). However, when running mocha, I’m getting an error indicating that the API server could not be reached, which suggests that the real requests.makeCallToAPI() is being called, and not my stub.
I must be missing something. What am I doing wrong here?
Where are you requiring the request.js? You will need to require request.js before you load up the module you want to test.
Edit 1: Using sinon.js
Here is a gist of what I meant: https://gist.github.com/limianwang/1114249de99c6a189384
Edit 2: Using proxyquire
If you are intending to test simply the utilities without concern of what actually happens within the requests.makeAPICall, you can use something like proxyquire to do the trick. If you are concerned with the actual logic within requests.js, you can use sinon.stub to stub out the actual request.get api.

Async call in node.js vs. mongoose

I have a node.js app that uses mongoose to connect to
a mongodb; i need to select all the documents inserted and i
i've problems with async stuff.
I've made a model with the following function:
exports.listItems=function() {
Ticket.find({}, function(err,tkts) {
console.log(tkts);
return tkts;
});
}
I correctly see the value of "tkts", but when i call it from:
exports.list = function(req,res) {
var items=db.listItems();
console.log("Items:"+items);
res.render('list', { title: title, items:items });
}
defined in app.js as:
app.get('/list', routes.list);
items is undefined (i think because of non-async definition of db.list()).
What am i doing wrong and how can it be corrected?
You need to use callbacks more appropriately.
A more traditional listItems function would be
exports.listItems = function(done) {
Ticket.find({}, done);
}
Then, in list, you could do:
exports.list = function(req,res) {
db.listItems(function(err,items){
console.log("Items:"+items);
res.render('list', { title: title, items:items });
});
}
Because of the asynchronous nature of Node.JS, you should always pass (and expect) a callback in your functions. So that you can defer execution if something asynchronous is executed.
Also: be sure to check out async, its an insanely good and easy-to-use library, that will simplify complex async scenarios in a breeze.

Resources