Asynchronous http calls with nodeJS - node.js

I would like to launch asynchronous http calls on my server node, i saw the async node module and i guess the async.parallel enables us to do that.
The documented example is pretty clear, but i don't know how i could manage multiple http calls.
I tried the example bellow but it doesn't even launch the http calls:
var http = require('http');
var Calls = [];
Calls.push(function(callback) {
// First call
http.get('http://127.0.0.1:3002/first' callback);
});
Calls.push(function(callback) {
// Second call
http.get('http://127.0.0.1:3002/second' callback);
});
var async = require('async');
async.parallel(Calls, function(err, results) {
console.log('async callback: '+JSON.stringify(results));
res.render('view', results);
});
If i launch the http requests separately, i do have a result, but but calling the async callback i get async callback: [null,null]

Have a look at the documentation:
With http.request() one must always call req.end() to signify that
you're done with the request - even if there is no data being written
to the request body.
You are creating a request, but you are not finalizing it. In your calls you should do:
var req = http.request(options, function(page) {
// some code
});
req.end();
This is assuming you are doing a normal GET request without body.
You should also consider using http.get which is a nice shortcut:
http.get("http://127.0.0.1:3002/first", function(res) {
// do something with result
});
Update The other thing is that callbacks in async have to be of the form
function(err, res) { ... }
The way you are doing it now won't work, because callback to http.get accepts only one argument res. What you need to do is the following:
http.get('http://127.0.0.1:3002/second', function(res) {
callback(null, res);
});

Ok the thing is to call the callback this way callback(null, res); instead callback(res);, i think the first parameter is interpreted as an error and the second one is the real result.

dont use capital names for other purpouses than types/classes
below is your code with corrected obvious mistakes
var http = require('http');
var calls = [];
calls.push(function(callback) {
// First call
http.get('http://127.0.0.1:3002/first', function (resource) {
resource.setEncoding('utf8');
resource.on('data', function (data) {
console.log('first received', data);
callback();
});
});
});
calls.push(function(callback) {
// Second call
http.get('http://127.0.0.1:3002/second', function (resource) {
resource.setEncoding('utf8');
resource.on('data', function (data) {
console.log('second received', data);
callback();
});
});
});
var async = require('async');
async.parallel(calls, function(err, results) {
console.log('async callback ', results);
res.render('view', results);
});

Related

Making a request before all tests start in Mocha

I would like to test my simple API that has /groups URL.
I want to make an API request to that URL (using Axios) before all tests begin and make the response visible to all test functions.
I am trying to make the response visible but not able to make it work. I followed a similar case with filling out the DB upfront but no luck with my case.
My simple test file below:
var expect = require('chai').expect
var axios = require('axios')
var response = {};
describe('Categories', function() {
describe('Groups', function() {
before(function() {
axios.get(config.hostname + '/groups').then(function (response) {
return response;
})
});
it('returns a not empty set of results', function(done) {
expect(response).to.have.length.greaterThan(0);
done();
})
});
});
I tried also a sligh modification of before function:
before(function(done) {
axios.get(config.hostname + '/groups')
.then(function (response) {
return response;
}).then(function() {
done();
})
});
but no luck too.
The error I am getting is simply that response isn't changing nor is visible within it. AssertionError: expected {} to have property 'length'
Summarising: How can I pass response from axios inside to in()?
Your first form is incorrect, because you're not returning the chained promise. As such, mocha has no way of knowing when your before is finished, or even that it's async at all. Your second form will solve this problem, but since axios.get already returns a promise, it's kind of a waste not to use mocha's built-in promise support.
As for making the response visible in the it, you need to assign it to a variable in a scope that will be visible within the it.
var expect = require('chai').expect
var axios = require('axios')
var response;
describe('Categories', function() {
describe('Groups', function() {
before(function() {
// Note that I'm returning the chained promise here, as discussed.
return axios.get(config.hostname + '/groups').then(function (res) {
// Here's the assignment you need.
response = res;
})
});
// This test does not need the `done` because it is not asynchronous.
// It will not run until the promise returned in `before` resolves.
it('returns a not empty set of results', function() {
expect(response).to.have.length.greaterThan(0);
})
});
});

node.js server and AWS asynchronous call issue

I have a simple node Express app that has a service that makesa call to a node server. The node server makes a call to an AWS web service. The AWS simply lists any S3 buckets it's found and is an asynchronous call. The problem is I don't seem to be able to get the server code to "wait" for the AWS call to return with the JSON data and the function returns undefined.
I've read many, many articles on the web about this including promises, wait-for's etc. but I think I'm not understanding the way these work fully!
This is my first exposer to node and I would be grateful if somebody could point me in the right direction?
Here's some snippets of my code...apologies if it's a bit rough but I've chopped and changed things many times over!
Node Express;
var Httpreq = new XMLHttpRequest(); // a new request
Httpreq.open("GET","http://localhost:3000/listbuckets",false);
Httpreq.send(null);
console.log(Httpreq.responseText);
return Httpreq.responseText;
Node Server
app.get('/listbuckets', function (req, res) {
var bucketData = MyFunction(res,req);
console.log("bucketData: " + bucketData);
});
function MyFunction(res, req) {
var mydata;
var params = {};
res.send('Here are some more buckets!');
var request = s3.listBuckets();
// register a callback event handler
request.on('success', function(response) {
// log the successful data response
console.log(response.data);
mydata = response.data;
});
// send the request
request.
on('success', function(response) {
console.log("Success!");
}).
on('error', function(response) {
console.log("Error!");
}).
on('complete', function() {
console.log("Always!");
}).
send();
return mydata;
}
Use the latest Fetch API (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) to make HTTP calls. It has built-in support with Promise.
fetch('http://localhost:3000/listbuckets').then(response => {
// do something with the response here
}).catch(error => {
// Error :(
})
I eventually got this working with;
const request = require('request');
request(url, function (error, response, body) {
if (!error && response.statusCode == 200) {
parseString(body, function (err, result) {
console.log(JSON.stringify(result));
});
// from within the callback, write data to response, essentially returning it.
res.send(body);
}
else {
// console.log(JSON.stringify(response));
}
})

node http-proxy: async modification of request body

I need to modify the request body asynchronously. Something along the lines of this:
proxy.on('proxyReq', function(proxyReq, req, res, options) {
if(req.body) {
new Promise(function(resolve){
setTimeout(function() { // wait for the db to return
'use strict';
req.body.text += 'test';
let bodyData = JSON.stringify(req.body);
proxyReq.setHeader('Content-Type','application/json');
proxyReq.setHeader('Content-Length', Buffer.byteLength(bodyData));
// stream the content
proxyReq.write(bodyData);
resolve();
},1);
});
}
});
When I run this I get the error saying cannot modfiy headers once they have been set. Which makes sense.
How can I halt the sending of the request until I'm ready? I've looked at removing various listeners from proxyReq without success..
By looking at the source code #-) it seems like it's not really possible because the proxyReq event is sent and then the code moves on.
If it would instead wait for a promise, it would be possible (if you'd return that promise as well).
A minimal fork on this lib could be for example:
// Enable developers to modify the proxyReq before headers are sent
proxyReq.on('socket', function(socket) {
if(server) { server.emit('proxyReq', proxyReq, req, res, options); }
});
(proxyReq.proxyWait || Promise.resolve())
.then( ... // rest of the code inside the callback
And then
proxy.on('proxyReq', function(proxyReq, req, res, options) {
if(req.body) {
proxyReq.proxyWait = new Promise(function(resolve){
setTimeout(function() { ...
But depending on your use case, there might be other solutions as well. For example, consider if it's really necessary that you use this proxy library. It You could alternatively use http directly, where you have all the control on the events and callbacks.
You can set selfHandleResponse: true inside the HttpProxy.createProxyServer. This then allows (and forces) you to handle the proxyRes manually!
const proxy = HttpProxy.createProxyServer({selfHandleResponse: true});
proxy.on('proxyRes', async (proxyReq, req, res, options) => {
if (proxyReq.statusCode === 404) {
req.logger.debug('Proxy Request Returned 404');
const something = await doSomething(proxyReq);
return res.json(something);
}
return x;// return original proxy response
});
I came here looking for the solution to a slightly different problem: Modifying the request headers (not body) before proxying.
I post this here in case that it is helpful to others. And maybe the code can be adapted to also modify the request body.
const http = require('http');
const httpProxy = require('http-proxy');
var proxy = httpProxy.createProxyServer({});
var server = http.createServer(function(req, res) {
console.log(`${req.url} - sleeping 1s...`);
setTimeout(() => {
console.log(`${req.url} - processing request`);
req.headers['x-example-req-async'] = '456';
proxy.web(req, res, {
target: 'http://127.0.0.1:80'
});
}, 1000);
});
server.listen(5050);

How to mock seneca calls in Node.js using jasmine-node?

I am using express module and below is the code of app.js
app.post('/test_url', function(request, response){
seneca.client({type: 'http',port: '3000',host: 'localhost',protocol: 'http'}).act({role: 'sample_role', cmd: 'save',firstname: request.params.firstname}, function (err, result) {
console.log("Inside Seneca act");
response.json(result);
})
});
Below is the test file where I am writing the test case for above code.
describe("POST /test_url/:firstname", function() {
it("should return status code 200", function(done) {
<b>//here I want to mock the call for seneca.client so that I can test if the call has been made with the required parameters.</b>
<b>//Also I would like to use the above mock object to further mock the call for act so that I can check if the act method has been called with the required parameters.'</b>
//Main purpose behind doing so is that I do not want the seneca methods to get actually called, and only want to test if the call has been made.
request.post("http://localhost:3000/test_url/sara", function(error, response, body) {
//some verification method on the mock object so as to confirm that both the calls i.e 'seneca.client' and 'seneca.client().act' have been called with the appropriate parameters
expect(body).toContain("success");
done();
});
});
});
I tried to mock the seneca calls using jasmine spy and sinon but still the call was actually being going to the method and the the callback function was also invoked resulting in the console.log("Inside Seneca act"); being called, which is not what I expect.
describe("POST /test_url/:firstname", function() {
it("should return status code 200", function(done) {
var senecaCall = sinon.stub(seneca, 'client');
//or spyOn(seneca, "client");
request.post("http://localhost:3000/test_url/sara", function(error, response, body) {
expect(body).toContain("success");
done();
});
});
});

Testing a function which consumes promises

I've got a MEAN app and I'm trying to get tests to work on the node side. Async events are wrapped in promises, which are consumed in the controller. I failed at testing the controller :(
The controller I'm trying to test:
ProjectController.prototype.getAll = function(req, res, next) {
req.dic.subjectRepository
.getById(req.params.subjectId)
.then(function(subject) {
res.json(subject.projects);
}, function(err) {
return res.status(404).send('Subject does not exist.' + err);
});
};
The subjectRepository is our data source, which returns a promise (mpromise because under the hood we're using mongoose, but it shouldn't matter):
So in our test we tried mocking the request (we're injecting our dependency injection container from a middleware into the req) and response (the test succeeds if response.json() has been called with the subjects we tried to fetch) and our subjectRepository. We used bluebird (although I tried others out of frustration) to create fake promises for our mocked subjectRepository:
describe('SubjectController', function() {
'use strict';
var Promise = require('bluebird');
it('gets all existing subjects', function() {
// -------------------------------------
// subjectRepository Mock
var subjectRepository = {
getAll: function() {},
};
var subjectPromise = Promise.resolve([
{name: 'test'},
{name: 'test2'},
]);
spyOn(subjectRepository, 'getAll').andReturn(subjectPromise);
// -------------------------------------
// request mock
var req = {
dic: {
subjectRepository: subjectRepository,
},
};
// -------------------------------------
// response mock
var res = {
json: function() {},
send: function() {},
};
spyOn(res, 'json');
// -------------------------------------
// actual test
var subjectController = new (require('../../../private/controllers/SubjectController'))();
subjectController.getAll(req, res);
// this succeeds
expect(subjectRepository.getAll).toHaveBeenCalled();
// this fails
// expect(res.json).toHaveBeenCalled();
});
});
Question: How do I make the test run the expect() AFTER the promise succeeded?
Node v0.12
The code is on GitHub for anyone who's interested: https://github.com/mihaeu/fair-projects
And maybe I should mention that the controller is called from the router:
// router handles only routing
// and controller handles data between view and model (=MVC)
subjectRouter.get('/:subjectId', subjectController.get);
I got this to work by changing our controllers to hand down the promises, but I'm not sure this is what we want. Isn't there a way to get my approach to work?
it('gets all existing subjects', function(done) {
// ...
var subjectController = new (require('../../../private/controllers/SubjectController'))();
subjectController.getAll(req, res).then(function() {
expect(res.json).toHaveBeenCalledWith(testSubjects); // success
}).finally(done);
expect(subjectRepository.getAll).toHaveBeenCalled(); // success
}
Your code makes the mistake of mixing business logic with front facing routing.
If your getAll did not touch the request and response object, it would look something like this:
ProjectController.prototype.getAll = function(subjectId) {
return req.dic.subjectRepository.getById(subjectId).then(function(subject){
return subject.projects;
});
};
Now, it is no longer related to the request response life cycle or in charge of logic, testing it is trivial by:
it("does foo", function(){
// resolve to pass the test, reject otherwise, mocha or jasmine-as-promised
return controller.getAll(152).then(...)
});
That would make your actual handler look like:
app.get("/projects", function(req, res){
controller.getAll(req.params.subjectId).then(function(result){
res.json(result);
}, function(){
res.status(404).send("...");
});
});

Resources