Our application consists of nodejs, express, reactjs, and newforms.
To make rest calls we are using :
var RestClient = require('superagent-ls')
And we are making rest calls like:
cleanBirthDate(callback) {
var {birthDate} = this.cleanedData
var formattedDob = moment (birthDate).format('DDMMYYYY')
RestClient.get(Global.getBirthDateServiceUrl() + '/' + formattedDob)
.end((err, res) => {
if (err) {
callback (err)
}
else if (res.clientError) {
var message = errorsMappingSwitch(res.body.error)
callback(null, forms.ValidationError(message))
}
else {
callback(null)
}
})
},
We want to move the RestClient related code to our own file say RestCleint.js and then require it and use it across the application. By doing so we can apply some generalised code(like error handling, logging, redirect to specific error pages depending on the error code) in one place.
Appreciate any help in this direction.
I did the exact same thing you require (even with using superagent). I created modules with the API code in a /utils folder and required them where applicable. For even more abstraction we're using CoffeeScript to create classes that inherit from a BaseAPIObject and invoke using something like API.Posts.getAll().end() etc.
This article was very helpful in understanding how to write your own modules: Export This: Interface Design Patterns for Node.js Modules.
you can always require it like
RestClient.js
export default function callApi(callback) {
//your rest code
// use the callback here in the callback of your call.
}
app.js
import {callApi} from './RestClient';
callApi((err, result) => {
if (err) console.log(err)
});
Related
I am looking into solutions to call some C# code from within my NodeJS application. I came across EdgeJS, which seems to make this possible. However, I came across part of their code that is confusing to me, because I see an async keyword without an accompanying await. And while I'm more familiar with JS than C#, my understanding is that in BOTH you need to include an await with any async. This is the code in question, where some multi-line C# code is defined within backticks:
var edge = require('edge');
var helloWorld = edge.func(`
async (input) => {
return ".NET Welcomes " + input.ToString();
}
`);
helloWorld('JavaScript', function (error, result) {
if (error) throw error;
console.log(result);
});
Can someone explain how this is working, considering await doesn't appear anywhere?
It must be simple but I can not find a good solution.
In a node module for example I want to log if something goes wrong. So I need want to use the Hapi function server.log(...). An example:
In the server.js:
const server = new Hapi.Server({...});
server.connection({port:3000});
server.start((err) => {
if (err) { throw err }
}
In the file test.js, I want to use the server variable created in the server.js. It should be usable in any module, because of logging and loading other plugins.
class Test {
logInfo(text) {
server.log(['info'], text); // this will not work.
}
}
module.exports = new Test();
I can ofcause add the server to every call for the module, but that would be an overkill.
How do I get a reference to the server variable outside the module it was created?
If you want to log a message during a request, you can use request.log(...).
Alternatively, you can use server.log(...) by accessing the server object on the request object in a handler.
{
method: 'GET',
path: '/do/stuff',
handler: function (request, reply) {
request.server.log('server.log() method');
request.log('request.log() method');
const test = new Test(request.server);
test.logInfo('some text here');
}
}
Check out my answer on logging in hapi, too.
In your case, you'd need to pass a reference to the server object to your module if you specifically wanted to log a message a hapi server event.
class Test {
private server;
constructor(server) {
this.server = server;
}
logInfo(text) {
this.server.log(['info'], text); // this will work.
}
}
But the commenter on your question is likely right. You should avoid this pattern if you want to avoid tight coupling of your modules.
I'm learning node.js coming from a PHP background with a limited JavaScript level. I think I got over now the change of mindset implied by the asynchronous approach. And I love it.
But, as many others before me, I quickly understood the concrete meaning of the "pyramid of doom".
So I build these little 'dummy' route and view to understand how to properly use Async.js. I just spend the last 5 hours writing the following code (rewritten of course tens of times). It works, but I wonder how I could go further and made this code more simple (less verbose, easier to read and maintain).
I found many resources on the web and especially here, but always by bits of info here and there.
I'm guessing at this point that I should use "bind" and "this" with async.apply to make to shorten the 2 last functions called by the waterfall.
The issue is to get the object "db" defined so I can use the "collection" method on it (for the second function).
I really searched an example in Google, but it's surprising that you don't get straightforward examples looking for "async waterfall bind" (as well as many keyword variations I tried). There are answers of course but none seems relevant to this particular issue... ore, quite possibly, I haven't understood them.
Can someone help me on this? I'll be quite grateful.
app.get('/dummy',
function(req, res) {
var MongoClient = require('mongodb').MongoClient;
async.waterfall(
[
async.apply(MongoClient.connect, 'mongodb://localhost:27017/mybdd'),
function(db, callback) {
db.collection('myCollection', callback);
},
function(collection, callback) {
collection.find().sort({"key":-1}).limit(10).toArray(callback);
}
], function(err, results) {
if (err) console.log('Error :', err);
else { res.render('dummy.jade', { title:'dummy', results: results} ); }
}
);
}
);
If you're using the mongodb JS Driver, then this should work:
async.waterfall(
[
function (cb) {
new MongoClient(...)
.connect('mongodb://localhost:27017/mybdd', cb);
},
function (db, callback) {
db.collection('myCollection', callback);
},
...
Alternatively, if you want to use async.apply, just pass an instance of MongoClient
async.apply(new MongoClient(...).connect, 'mongodb://localhost:27017/mybdd')
I've recently created a simple abstraction named WaitFor to call async functions in sync mode (based on Fibers): https://github.com/luciotato/waitfor
I'm not familiar with mongodb client, so i'll be mostly guessing what you're trying to do:
using WaitFor your code will be:
var MongoClient = require('mongodb').MongoClient;
var wait = require('waitfor');
app.get('/dummy', function(req, res) {
// handle request in a Fiber, keep node spinning
wait.launchFiber(handleDummy,req,res)
}
);
function handleDummy(req, res) {
try {
var db = wait.for(MongoClient.connect, 'mongodb://localhost:27017/mybdd');
var collection = wait.forMethod(db,'collection','myCollection');
var results = wait.forMethod(collection.,'sort',{"key":-1}).toArray();
res.render('dummy.jade', { title:'dummy', results: results} );
}
catch(err) {
res.render('error.jade', { title:'error', message: err.message} );
}
};
I'm using Mongoose with Node.js and have the following code that will call the callback after all the save() calls has finished. However, I feel that this is a very dirty way of doing it and would like to see the proper way to get this done.
function setup(callback) {
// Clear the DB and load fixtures
Account.remove({}, addFixtureData);
function addFixtureData() {
// Load the fixtures
fs.readFile('./fixtures/account.json', 'utf8', function(err, data) {
if (err) { throw err; }
var jsonData = JSON.parse(data);
var count = 0;
jsonData.forEach(function(json) {
count++;
var account = new Account(json);
account.save(function(err) {
if (err) { throw err; }
if (--count == 0 && callback) callback();
});
});
});
}
}
You can clean up the code a bit by using a library like async or Step.
Also, I've written a small module that handles loading fixtures for you, so you just do:
var fixtures = require('./mongoose-fixtures');
fixtures.load('./fixtures/account.json', function(err) {
//Fixtures loaded, you're ready to go
};
Github:
https://github.com/powmedia/mongoose-fixtures
It will also load a directory of fixture files, or objects.
I did a talk about common asyncronous patterns (serial and parallel) and ways to solve them:
https://github.com/masylum/i-love-async
I hope its useful.
I've recently created simpler abstraction called wait.for to call async functions in sync mode (based on Fibers). It's at an early stage but works. It is at:
https://github.com/luciotato/waitfor
Using wait.for, you can call any standard nodejs async function, as if it were a sync function, without blocking node's event loop. You can code sequentially when you need it.
using wait.for your code will be:
//in a fiber
function setup(callback) {
// Clear the DB and load fixtures
wait.for(Account.remove,{});
// Load the fixtures
var data = wait.for(fs.readFile,'./fixtures/account.json', 'utf8');
var jsonData = JSON.parse(data);
jsonData.forEach(function(json) {
var account = new Account(json);
wait.forMethod(account,'save');
}
callback();
}
That's actually the proper way of doing it, more or less. What you're doing there is a parallel loop. You can abstract it into it's own "async parallel foreach" function if you want (and many do), but that's really the only way of doing a parallel loop.
Depending on what you intended, one thing that could be done differently is the error handling. Because you're throwing, if there's a single error, that callback will never get executed (count won't be decremented). So it might be better to do:
account.save(function(err) {
if (err) return callback(err);
if (!--count) callback();
});
And handle the error in the callback. It's better node-convention-wise.
I would also change another thing to save you the trouble of incrementing count on every iteration:
var jsonData = JSON.parse(data)
, count = jsonData.length;
jsonData.forEach(function(json) {
var account = new Account(json);
account.save(function(err) {
if (err) return callback(err);
if (!--count) callback();
});
});
If you are already using underscore.js anywhere in your project, you can leverage the after method. You need to know how many async calls will be out there in advance, but aside from that it's a pretty elegant solution.
So Im trying to use the nodejs express FS module to iterate a directory in my app, store each filename in an array, which I can pass to my express view and iterate through the list, but Im struggling to do so. When I do a console.log within the files.forEach function loop, its printing the filename just fine, but as soon as I try to do anything such as:
var myfiles = [];
var fs = require('fs');
fs.readdir('./myfiles/', function (err, files) { if (err) throw err;
files.forEach( function (file) {
myfiles.push(file);
});
});
console.log(myfiles);
it fails, just logs an empty object. So Im not sure exactly what is going on, I think it has to do with callback functions, but if someone could walk me through what Im doing wrong, and why its not working, (and how to make it work), it would be much appreciated.
The myfiles array is empty because the callback hasn't been called before you call console.log().
You'll need to do something like:
var fs = require('fs');
fs.readdir('./myfiles/',function(err,files){
if(err) throw err;
files.forEach(function(file){
// do something with each file HERE!
});
});
// because trying to do something with files here won't work because
// the callback hasn't fired yet.
Remember, everything in node happens at the same time, in the sense that, unless you're doing your processing inside your callbacks, you cannot guarantee asynchronous functions have completed yet.
One way around this problem for you would be to use an EventEmitter:
var fs=require('fs'),
EventEmitter=require('events').EventEmitter,
filesEE=new EventEmitter(),
myfiles=[];
// this event will be called when all files have been added to myfiles
filesEE.on('files_ready',function(){
console.dir(myfiles);
});
// read all files from current directory
fs.readdir('.',function(err,files){
if(err) throw err;
files.forEach(function(file){
myfiles.push(file);
});
filesEE.emit('files_ready'); // trigger files_ready event
});
As several have mentioned, you are using an async method, so you have a nondeterministic execution path.
However, there is an easy way around this. Simply use the Sync version of the method:
var myfiles = [];
var fs = require('fs');
var arrayOfFiles = fs.readdirSync('./myfiles/');
//Yes, the following is not super-smart, but you might want to process the files. This is how:
arrayOfFiles.forEach( function (file) {
myfiles.push(file);
});
console.log(myfiles);
That should work as you want. However, using sync statements is not good, so you should not do it unless it is vitally important for it to be sync.
Read more here: fs.readdirSync
fs.readdir is asynchronous (as with many operations in node.js). This means that the console.log line is going to run before readdir has a chance to call the function passed to it.
You need to either:
Put the console.log line within the callback function given to readdir, i.e:
fs.readdir('./myfiles/', function (err, files) { if (err) throw err;
files.forEach( function (file) {
myfiles.push(file);
});
console.log(myfiles);
});
Or simply perform some action with each file inside the forEach.
I think it has to do with callback functions,
Exactly.
fs.readdir makes an asynchronous request to the file system for that information, and calls the callback at some later time with the results.
So function (err, files) { ... } doesn't run immediately, but console.log(myfiles) does.
At some later point in time, myfiles will contain the desired information.
You should note BTW that files is already an Array, so there is really no point in manually appending each element to some other blank array. If the idea is to put together the results from several calls, then use .concat; if you just want to get the data once, then you can just assign myfiles = files directly.
Overall, you really ought to read up on "Continuation-passing style".
I faced the same problem, and basing on answers given in this post I've solved it with Promises, that seem to be of perfect use in this situation:
router.get('/', (req, res) => {
var viewBag = {}; // It's just my little habit from .NET MVC ;)
var readFiles = new Promise((resolve, reject) => {
fs.readdir('./myfiles/',(err,files) => {
if(err) {
reject(err);
} else {
resolve(files);
}
});
});
// showcase just in case you will need to implement more async operations before route will response
var anotherPromise = new Promise((resolve, reject) => {
doAsyncStuff((err, anotherResult) => {
if(err) {
reject(err);
} else {
resolve(anotherResult);
}
});
});
Promise.all([readFiles, anotherPromise]).then((values) => {
viewBag.files = values[0];
viewBag.otherStuff = values[1];
console.log(viewBag.files); // logs e.g. [ 'file.txt' ]
res.render('your_view', viewBag);
}).catch((errors) => {
res.render('your_view',{errors:errors}); // you can use 'errors' property to render errors in view or implement different error handling schema
});
});
Note: you don't have to push found files into new array because you already get an array from fs.readdir()'c callback. According to node docs:
The callback gets two arguments (err, files) where files is an array
of the names of the files in the directory excluding '.' and '..'.
I belive this is very elegant and handy solution, and most of all - it doesn't require you to bring in and handle new modules to your script.