What are the consequences of making request global (or singleton), so that it is accessible all over the server and does not have to be passed in each function call? For example:
index.js:
const http = require('http');
const { saveReq } = require('./shared');
const {
doSomethingWithReqPassingItAsParameter,
doSomethingWithReqPassingItAsGlobal
} = require('./lib');
const requestListener = function (req, res) {
// approach 1
doSomethingWithReqPassingItAsParameter(req);
// approach 2
saveReq(req);
doSomethingWithReqPassingItAsGlobal();
res.writeHead(200);
res.end('Hello, World!');
}
const server = http.createServer(requestListener);
server.listen(8080);
lib.js:
const { loadReq } = require('./shared');
const doSomethingWithReqPassingItAsParameter = (req) => {
console.log('req as parameter', req.url);
};
const doSomethingWithReqPassingItAsGlobal = () => {
console.log('req as global', loadReq().url);
};
module.exports = {
doSomethingWithReqPassingItAsParameter,
doSomethingWithReqPassingItAsGlobal,
};
shared.js
var request;
const saveReq = (r) => request = r;
const loadReq = () => request;
module.exports = {
saveReq,
loadReq,
}
This is very convenient for large projects with many levels of function calls, but how parallel requests will be handled? I know that nodejs is single-threaded, does it mean than each http request will be run from end to finish separately or they can overlap, thus using a global request object would make a mess?
The consequences are that your server will only work for one request at a time and as soon as you have more than one request in flight at the same time, data will be mixed up between requests dealing to bugs, crashes, security issues and incorrect results.
Simply put, you cannot program a server that way. Pass the req object or data from it to any functions that need it. That keeps the appropriate req object associated with the right execution to avoid all the problems of trying to store a req object in some sort of global location where multiple requests in flight at the same time will/can conflict.
There is a relatively new thing in nodejs called "async local storage" that could perhaps be used for this. You can read a little about it here, though it's my personal opinion that it's still better to pass your request data to the functions that want to use it rather than the async local storage for this.
Related
I know what is wrong with my code and I have looked into the best way of solving it, however with my lack of experience, I am having a hard time finding a good answer.
I need my first route(/data) to be fully completed before the second(/logo) express route sends the data. In short, I just need the variable symbolUrl to be completed before it goes into the second fetch call. Here is the code down below to explain
app.use(express.static('public'));
const url =
'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest';
const qString =
'?CMC_PRO_API_KEY=' + process.env.apiKey + '&start=1&limit=10&convert=USD';
let symbol = [];
app.get('/data', async (req, res) => {
const fetch_res = await fetch(url + qString);
const coinData = await fetch_res.json();
for (let i = 0; i < 9; i++) {
symbol.push(coinData.data[i]['symbol']);
};
res.json(coinData);
});
app.get('/logo', async (req, res) => {
const symbolUrl = symbol.join(',');
const url2 = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/info';
const qString2 = `?CMC_PRO_API_KEY=${apiKey}%symbol=${symbolUrl}`;
const fetch_res2 = await fetch(url2 + qString2);
const coinLogo = await fetch_res2.json();
res.json(coinLogo);
});
The issue I am trying to solve with this project is that I want to send the data(/data) to be sent to the front end first because this API call will load the majority of the page. Then my second call will load images and other larger files afterward. HOWEVER, the API I am working with to get the logos(images) of the specific crypto coins I want, I need a different endpoint as well as use %symbol=${symbolUrl} in the API call to get the correct tokens I want to call.
client code:
fetch('http://localhost:2000/data')
.then(async (response) => {
return response.json();
})
.then(async (data) => {
const parsedData = data['data'];
// console.log(data['data'][0]['name'])
await parsedData.forEach((element) => {
// this just has about 20 lines of code generating the the look of the page. It works as intended
});
fetch('http://localhost:2000/logo')
.then(async (response) => {
return response.json();
})
.then(async (logo) => {
console.log(logo)});
***I have tried putting this in an async function and awaiting the first fetch call
All I need to be done is for app.get(/data) to be fully complete before doing my second app.get. I have done testing and I know that is the issue. I apologize if it is something easy, but I couldn't find anything on making an app.get synchronous and I have tried putting both in a async function, however that did not work.
You cannot send responses in fragments like you're trying to do, it would throw an error saying Can't set headers after they are sent to client
The proper method to implement what you are trying to do is to define the first layer as middleware, and then allow the second layer to return the response. Here layer basically means a function handler.
In order to control when the execution passes to the next layer / next function handler, express has a third parameter (request, response, next). You're only using request and response, researching about next will solve your concern.
Express next function, what is it really for?
First handler
app.get('something_unique', async (req, res, next) => {
// do whatever you want to do first
// save data into res.locals
res.locals.foo = {...}
next()
})
Second Handler
app.get('something_unique', (req, res) => {
const data = res.locals.foo;
// whatever you want
return res.json({ anything })
})
More:
Express next function, what is it really for?
Error: Can't set headers after they are sent to the client
Passing variables to the next middleware using next() in Express.js
I'm not sure what client code you're really running as it sounds like you've bee trying several things, but this should work to sequence the /data request and the /logo request so that the /logo request is not run until the response from the /data request has been received.:
async function run() {
const r1 = await fetch('http://localhost:2000/data');
const data = await r1.json();
const parsedData = data.data;
parsedData.forEach((element) => {
// this just has about 20 lines of code generating
// the the look of the page. It works as intended
});
const r2 = await fetch('http://localhost:2000/logo');
const logo = await r2.json();
return logo;
}
run().then(logo => {
console.log(logo);
}).catch(err => {
// handle errors here
console.log(err);
});
If there is any asynchronous code inside the .forEach(), then we will have to see that also to properly sequence that.
As I've said in my comments, stuffing the data from the first request into a server-side variable is probably the wrong design on the server because two separate clients both issuing /data requests will conflict with one another, creating race conditions. But, you haven't explained what this data is really for or why you're stuffing it into a variable on the server for us to suggest an alternate design.
I have a website that will use EJS templating to display data from my database. Instead of creating a second API just to do this, I would like to use the pre-existing API that other developers can use. Is there a way to get data from my API in the most meta way possible.
main app route
app.get("/facts/random", async (req, res) => {
/* Make request to /api/random here and get JSON data */
});
API route
app.get("/api/random/", async (req, res) => {
let results = await Fact.find();
const randomResult = Math.floor(Math.random() * results.length);
return res.json(results[randomResult]);
})
I want to make a request from the first route to the second route using the most meta way possible.
If /api/random is in another microservice then you can use Axios to make an http call, but if it is in the same Express App then refactor the code and turn it into a function and call that function in both the controllers.
Usually, you don't have your own http server make a request to itself. Usually, you just factor out the functionality the other API is using into an exported function and you call the function directly.
It's definitely more efficient that way since there's really no reason to package it up into an http request, go through the network stack, parse it as an http request in your server, process it, create an http response, send that back through the network stack, parse the response, then process it when you could avoid 6 of those 8 steps by just calling a common function you factored out.
app.get("/facts/random", async (req, res) => {
let randomResult = await apiRandom();
...
});
async function apiRandom() {
let results = await Fact.find();
const randomResult = Math.floor(Math.random() * results.length);
return results[randomResult];
}
app.get("/api/random/", async (req, res) => {
let randomResult = await apiRandom();
res.json(randomResult);
});
If, for some reason, you do want to make requests to your own http server, then get a library like got() or axios() that makes that easy.
By my yet little understanding on how NodeJS/Javascript works, I believe it should be possible. Is it?
I would like to be able to send a function to a client based on http request, something like this (pseudo short code):
// Server
server.get("/function",()=>{
return function () => {
console.log('test)
}
// client
request ("server/function",(res)=>{
func = res.body
func()
// # result
// test
I have tried to convert to string to send the function from the server and convert back to an object on the client, but it return errors that I couldn't understand why as if I send a simple key value json object it works.
// server
const http = require('http');
const port = 3000;
const hostname = '127.0.0.1';
const server = http.createServer((req, res) => {
if (req.url === '/function') {
res.statusCode = 200;
function func () {
console.log('test')
}
res.end(func.toString())
}
});
server.listen(3000)
// client
const http = require('http');
const httpGet = url => {
return new Promise((resolve, reject) => {
http.get(url, res => {
res.setEncoding('utf8');
let body = '';
res.on('data', chunk => body += chunk);
res.on('end', () => resolve(body));
}).on('error', reject);
});
};
const call = async () => {
const body = await httpGet('http://localhost:3000/function')
console.log(`${body}`)
func = JSON.parse(body) // try to redefine function based on the response
}
const func = () => {
console.log('before')
}
func() // run the func defined here
call()
func() // run the function got from the request?
The idea is to have a client that will be able to execute almoust any code without the need of updating the client itself.
I'm not sure why you want to do this, it is really unsafe and you shouldn't have web applications that do this. But here goes:
Let's say you have a function:
function test() {
console.log("test")
}
And as string:
const funcString = test.toString()
// ^ is now "function test() {\n console.log(\"test\")\n }"
You need to first get the function body:
const funcBody = funcString.replace(/^[^{]+{/, '').replace(/}[^}]*$/, '')
// ^ is now "console.log(\"test\")"
Then create a function from it:
const newTest = Function(funcBody)
Now if you call newTest() you will see test printed in your console.
Note that the regular expression I have used to get the function body will not work on all functions. For example, for const test = () => 1 you will need a different regular expression.
I'm not sure this is the best idea.
Historically, the client-server relationship you are describing was inversed through remote procedure calls where the client would invoke a specific endpoint on a remote server. It sounds like your biggest draw towards having the client arbitrarily execute the code was the removal of updating the client code. What happens if you want to make backwards-incompatible changes to the server code? I think you will have better success using versioned API endpoints to execute code server-side based on client-side logic, which you will find many RPC and/or REST frameworks for within npm.
The ideia is to have a client that will be able to execute almoust any code without the need of updating the client itself.
One final thing to keep in mind are the security implications of this. What happens if I find your client and substitute in my malicious JavaScript?
I'm trying to write some tests using Lab and Sinon for various HTTP requests that are called in a file of mine. I followed the Fake XMLHttpRequest example at http://sinonjs.org/ but when I run my tests it appears to not actually capture any requests.
Here is the (relevant) testing code:
context('when provided a valid payload', function () {
let xhr;
let requests;
before(function (done) {
xhr = sinon.useFakeXMLHttpRequest();
requests = [];
xhr.onCreate = function (req) { requests.push(req); };
done();
});
after(function (done) {
// clean up globals
xhr.restore();
done();
});
it('responds with the ticket id', (done) => {
create(internals.validOptions, sinon.spy());
console.log(requests); // Logs empty array []
done();
});
});
create is the function I imported from the other file, here:
internals.create = async function (request, reply) {
const engineeringTicket = request.payload.type === 'engineering';
const urgentTicket = request.payload.urgency === 'urgent';
if (validation.isValid(request.payload)) {
const attachmentPaths = formatUploads(request.payload.attachments);
const ticketData = await getTicket(request.payload, attachmentPaths);
if (engineeringTicket) {
const issueData = getIssue(request.payload);
const response = await jira.createIssue(issueData);
jira.addAttachment(response.id, attachmentPaths);
if (urgentTicket) {
const message = slack.getMessage(response);
slack.postToSlack(message);
}
}
zendesk.submitTicket(ticketData, function (error, statusCode, result) {
if (!error) {
reply(result).code(statusCode);
} else {
console.log(error);
}
});
} else {
reply({ errors: validation.errors }).code(400); // wrap in Boom
}
};
as you can see it calls jira.createIssue and zendesk.submitTicket, both of which use an HTTP request to post some payload to an API. However, after running the test, the requests variable is still empty and seems to have captured no requests. It is definitely not actually submitting the requests as no tickets/issues have been created, what do I need to fix to actually capture the requests?
Your problem is apparent from the tags: you are running the code in NodeJS, but the networking stubs in Sinon is for XMLHttpRequest, which is a browser specific API. It does not exist in Node, and as such, the setup will never work.
That means if this should have worked you would have needed to run the tests in a browser. The Karma test runner can help you with this if you need to automate it.
To make this work in Node you can either go for an approach where you try to stub out at a higher level - meaning stubbing the methods of zendesk and jira, or you can continue with the approach of stubbing network responses (which makes the tests a bit more brittle).
To continue stubbing out HTTP calls, you can do this in Node using Nock. Saving the requests like you did above is done like this:
var requests = [];
var scope = nock('http://www.google.com')
.get('/cat-poems')
.reply(function(uri, requestBody) {
requests.push( {uri, requestBody} );
});
To get some insights on how you can stub out at a higher level, I wrote this answer on using dependency injection and Sinon, while this article by Morgan Roderick gives an intro to link seams.
My application has several layers: middleware, controllers, managers. Controllers interface is identical to middlewares one: (req, res, next).
So my question is: how can I test my controllers without starting the server and sending 'real' requests to localhost. What I want to do is to create request, response instances as nodejs does and then just call controllers method.
Something like this:
var req = new Request()
var res = new Response()
var next = function(err) {console.log('lala')}
controller.get_user(req, res, next)
Any advice is highly appreciated. Thanks!
P.S. the reason why I want to do this is that at the end I would like to test whether the response object contains correct variables for the jade views.
There's a semi decent implementation at node-mocks-http
Require it:
var mocks = require('node-mocks-http');
you can then compose req and response objects:
req = mocks.createRequest();
res = mocks.createResponse();
You can then test your controller directly:
var demoController = require('demoController');
demoController.login(req, res);
assert.equal(res.json, {})
caveat
There is at time of writing an issue in this implementation to do with the event emitter not being fired.
Since JavaScript is a dynamically typed language you can create mock objects and passing them to your controllers as follow:
var req = {};
var res = {};
var next = function(err) {console.log('lala')}
controller.get_user(req, res, next)
If your controller needs a particular piece of data or functionality from your request or response object you'll need to provide such data or functionality in your mocks. For example,
var req = {};
req.url = "http://google.com"; // fake the Url
var res = {};
res.write = function(chunk, encoding) {
// fake the write method
};
var next = function(err) {console.log('lala')}
controller.get_user(req, res, next)
I would try using dupertest for this. It's a node module I created for the very purpose of easy controller testing without having to spin up a new server.
It keeps the familiar syntax of node modules like request or supertest, but again, without the need to spin up a server.
It runs a lot like Hector suggested above, but integrates with a test framework like Jasmine to feel a little more seamless.
An example relating to your question may look like:
request(controller.get_user)
.params({id: user_id})
.expect(user, done);
Or the more explicit longhand version:
request(controller.get_user)
.params({id: user_id})
.end(function(response) {
expect(response).toEqual(user);
done();
});
Note: the examples assume user_id and user are defined somewhere, and that the controller grabs and returns a user based on id.
Edit: reading your response to an answer above, I will admit the downside currently is that this module does not integrate a more robust mock request or response object by default. dupertest makes it super easy to extend and add properties to both req and res, but by default they are pretty bare.
If you want to use the real req and res objects, you have to send real requests to the server. However this is much easier than you might think. There are a lot of examples at the express github repo. The following shows the tests for req.route
var express = require('../')
, request = require('./support/http');
describe('req', function(){
describe('.route', function(){
it('should be the executed Route', function(done){
var app = express();
app.get('/user/:id/edit', function(req, res){
// test your controllers with req,res here (like below)
req.route.method.should.equal('get');
req.route.path.should.equal('/user/:id/edit');
res.end();
});
request(app)
.get('/user/12/edit')
.expect(200, done);
})
})
})
A bit old post, but I would like to give my 2 cents. The approach you want to take depends on whether you are doing unit testing or integration testing. If you are going down the route of using supertest, that means you are running the actual implementation code and that means you are doing integration testing. If that's what you want to do this approach is fine.
But if you are doing unit testing, you would mock req and res objects (and any other dependencies involved). In the below code (non-relevant code removed for brevity), I am mocking res and giving just a mock implementation of json method, as that's the only method I need for my tests.
// SUT
kids.index = function (req, res) {
if (!req.user || !req.user._id) {
res.json({
err: "Invalid request."
});
} else {
// non-relevent code
}
};
// Unit test
var req, res, err, sentData;
describe('index', function () {
beforeEach(function () {
res = {
json: function (resp) {
err = resp.err;
sentData = resp.kids;
}
};
});
it("should return error if no user passed in request", function () {
req = {};
kidsController.index(req, res);
expect(err).to.equal("Invalid request.");
});
/// More tests....
})
Take a look at node-tdd and the useNock flag. It builds on top of mocha and nock and automatically creates and uses a recording file for each test.
We love that it's so easy to use. Basically just "enable and forget" and focus on writing requests / test cases. If requests for a test change, one still needs to delete or adjust the recording file, but at least it's entirely separate from the code.