Background
Yes, there are a lot of different Node.js logging library winston, bunyan and console.log. It's easy to log down the information of the specific request when it has called and when and what information would be in response.
The problem
The problem begins with the sub function calls. When under one request your calling multiple functions which also uses the same logging, how would you pass the request meta - data to these log calls (function parameters seems to be one possible way but these are really messy) ?
Example
Small visual for coders:
// Middleware to set some request based information
app.use(function (req, res, next) {
req.rid = 'Random generated request id for tracking sub queries';
});
app.get('/', function (req, rest) {
async.series({
'users': async.apply(db.users.find),
'posts': async.apply(db.posts.find),
}, function (err, dbRes) {
console.log('API call made ', req.rid)
res.end(dbRes);
});
});
// Now the database functions are in other file but we also need to track down the request id in there
(db.js)
module.exports = {
users: {
find: function () {
console.log('Calling users listing ', req.rid); // ERROR this is not possible to access, not in this scope
// Make query and return result
}
},
posts: {
find: function () {
console.log('Calling post listing ', req.rid); // ERROR this is not possible to access, not in this scope
// Make query and return result
}
}
};
You can log your requests with simple conf in your app.js with;
app.use(function(req, res, next){
console.log('%s %s', req.method, req.url);
next();
});
However, you need to provide logs for specific functions in your controller.
Related
app.route('/users')
.post(user.post)
.get(user.get)
.get(user.everyone)
.put(user.update)
.delete(user.delete);
I have ran into the problem of my function using two res.send, so I am getting the 'Error: cannot set header after they are sent.' error, to fix this I have turned it into two functions which I am trying to use two .get on the app.route, but it seems I can only use one as when I use two the second one doesn't work.
Is there a way I could use two .get on one app.route?
If not, what are my options to get around this problem?
You need to create separate routes for each api endpoint like this:
app.route('/users').get(req, res) => {
//Get all users
});
app.route('/users:/id').get(req, res) => {
//Get specific user
});
app.route('/users').post(req, res) => {
//Create new user
});
app.route('/users/:id').put(req, res) => {
//Update user
});
app.route('/users/:id').delete(req, res) => {
//Delete user
});
Once res.send is called, means your server has sent the response to the browser or whatever. you can't change the already sent response and its header.
You can use multiple callbacks on one route and one method(post,get)
An array of callback functions can handle a route. For example:
var cb0 = function (req, res, next) {
console.log('CB0')
next()
}
var cb1 = function (req, res, next) {
console.log('CB1')
next()
}
var cb2 = function (req, res) {
res.send('Hello from C!')
}
app.get('/example/c', [cb0, cb1, cb2])
yes, you can use multiple HTTP requests either its .get or .post but with different params. or routes.
I have an API POST route where I receive data from a client and upload the data to another service. This upload is done inside of the post request (async) and takes awhile. The client wants to know their post req was received prior to the async (create project function) is finished. How can I send without ending the POST? (res.send stops, res.write doesn't send it out)
I thought about making an http request back to their server as soon as this POST route is hit. . .
app.post('/v0/projects', function postProjects(req, res, next) {
console.log('POST notice to me');
// *** HERE, I want to send client message
// This is the async function
createProject(req.body, function (projectResponse) {
projectResponse.on('data', function (data) {
parseString(data.toString('ascii'), function (err, result) {
res.message = result;
});
});
projectResponse.on('end', function () {
if (res.message.error) {
console.log('MY ERROR: ' + JSON.stringify(res.message.error));
next(new Error(res));
} else {
// *** HERE is where they finally receive a message
res.status(200).send(res.message);
}
});
projectResponse.on('error', function (err) {
res.status(500).send(err.message);
});
});
});
The internal system requires that this createProject function is called in the POST request (needs to exist and have something uploaded or else it doesn't exist) -- otherwise I'd call it later.
Thank you!
I think you can't send first response that post request received and send another when internal job i.e. createProject has finished no matter success or fail.
But possibly, you can try:
createProject(payload, callback); // i am async will let you know when done! & it will push payload.jobId in doneJobs
Possibility 1, If actual job response is not required:
app.post('/v0/projects', function (req, res, next) {
// call any async job(s) here
createProject(req.body);
res.send('Hey Client! I have received post request, stay tuned!');
next();
});
});
Possibility 2, If actual job response is required, try maintaining queue:
var q = []; // try option 3 if this is not making sense
var jobsDone = []; // this will be updated by `createProject` callback
app.post('/v0/projects', function (req, res, next) {
// call async job and push it to queue
let randomId = randomId(); // generates random but unique id depending on requests received
q.push({jobId: randomId });
req.body.jobId = randomId;
createProject(req.body);
res.send('Hey Client! I have received post request, stay tuned!');
next();
});
});
// hit this api after sometime to know whether job is done or not
app.get('/v0/status/:jobId', function (req, res, next) {
// check if job is done
// based on checks if done then remove from **q** or retry or whatever is needed
let result = jobsDone.indexOf(req.params.jobId) > -1 ? 'Done' : 'Still Processing';
res.send(result);
next();
});
});
Possibility 3, redis can be used instead of in-memory queue in possibility 2.
P.S. There are other options available as well to achieve the desired results but above mentioned are possible ones.
I have built , using express() , a variety of methods. for simplicity let's I assume I built 2 POST() functions and I want to be able to use them by themselves and also to concatenate them via middleware for combine usage.
app.post('/create_obj_1' , function (req,res) {
//create Object_type_1
// send Object_type_1 via EXTERNAL API to somewhere
res.json({{ "statusCode": 200, "message": "OK" }
}
app.post('/create_obj_2' , function (req,res) {
//create Object_type_2
// send Object_type_2 via EXTERNAL API to somewhere
res.json({{ "statusCode": 200, "message": "OK" }
}
I want to have a new POST() that can invoke both of the other 2 (but still support stand alone invoking of the original 2
I think it's possible via middleware but I am not sure how - this is how I thought the new POST() should look like -
app.post('/create_obj_all' , function (req,res) {
//I want to invoke the create_obj_1 & create_obj_2 , check all OK, and finish
res.json({{ "statusCode": 200, "message": "OK" }
}
I am not sure how to approach the middleware usage in such case.
On top - how can I connect them to use one each other res? let's say the EXTERNAL API returns some value from obj_1 creation which I want to use in obj_2 post() function..
a Pseudo code of my attempt to use request() inside the middlware_1 -
var middle_1 = function (req, res, next) {
req.middle_1_output = {
statusCode : 404,
message : "fail_1"
}
var options = {
method: 'PUT', url: `EXTERNAL_API`, headers:
{
'cache-control': 'no-cache',
'content-type': 'application/x-www-form-urlencoded',
apikey: `KEY`
}
};
request(options, function (error, response, body) {
if (error) throw new Error(error);
// CODE THAT DO SOMETHING AND GET INFORMATION
// OLD WAY OF res.send here , to allow using in post.POST() was - res.status(200).send(body);
//res.status(200).send(body);
req.middle_1_output.statusCode = 200;
req.middle_1_output.message = "hello world";
});
next(); // trigger next middleware
}
Given the current example, I don't think you can do it unless you tweak the middlewares for the first two routes a bit:
var middleware1 = function(req, res, next) {
//create Object_type_1
// send Object_type_1 via EXTERNAL API to somewhere
next(); // calling next() triggers the next middleware
};
var middleware2 = function(req, res, next) {
//create Object_type_2
// send Object_type_2 via EXTERNAL API to somewhere
next(); // calling next() triggers the next middleware
};
/**
* This middleware is only used to send success response
*/
var response_success = function(req, res) {
res.json({ "statusCode": 200, "message": "OK" });
}
app.post('/create_obj_1', middleware1, response_success);
app.post('/create_obj_2', middleware2, response_success);
app.post('/create_obj_all', middleware1, middleware2, response_success);
Note that this is a very simplistic solution that I made from your example. The actual implementation will depend on what input each middleware is expecting and what output they generate. Also unlike here, there may also be different middlewares for sending the response.
2nd Part Addressing the second part of your question, if I have got you correctly you want to pass the output from middleware1 to middleware2. You can simply attach the output to the req object before calling next();. Like so:
var middleware1 = function(req, res, next) {
// do something
some_external_api_call(function(error, data) {
if (error) {
// handle the error yourself or call next(error);
} else {
req.middleware1_output = data; // set the output of the external api call into a property of req
next();
}
});
};
var middleware2 = function(req, res, next) {
// check to see if the middleware1_output has been set
// (meaning that the middleware has been called from /create_obj_all )
if (req.middleware1_output) {
// do something with the data
} else {
// handle scenario when /create_obj_2 is called by itself
}
next(); // calling next() triggers the next middleware
};
Notice how you have to account for both scenarios where middleware2 is called from POST /create_obj_all or directly from POST /create_obj_2.
3rd Part You should call next from within the callback. See my above example. This is due to the asynchronous/non-blocking nature of javascript.
function middleware(req, res, next) {
// do something
call_1st_external_api(some_options, function(error, data) {
// executed after call_1st_external_api completes
req.output_of_1st_external_api = data; // store the data of this api call for access from next middleware
next(); // calls the next middleware
// nothing here will be executed as next has already been called
});
// anything here will be executed before call_1st_external_api is completed
next(); // this will call the next middleware before call_1st_external_api completes
}
To handle two external APIs in the same middlewares you have to nest them (or use async or promises):
function middleware(req, res, next) {
// do something
call_1st_external_api(some_options, function(error1, data1) {
// executed after call_1st_external_api completes
req.output_of_1st_external_api = data1; // store the data of this api call for access from next middleware
// executed after call_2nd_external_api completes
call_2nd_external_api(some_options, function(error2, data2) {
req.output_of_2nd_external_api = data2; // store the data of this api call for access from next middleware
next();
});
// anything here will be executed before call_2nd_external_api is completed
});
// anything here will be executed before call_1st_external_api is completed
}
You have to handle all the errors above like I've shown in the 2nd Part which I have not shown in the above example for the sake of simplicity.
I am writing an app in node.js, I have the following code.
API for retrieving topic from DB
allTopics = function (req, res) {
db.Topic.all({limit: 10}).success(function (topics) {
res.send(topics)
});
};
Route for topics index
app.get('/topics', function (req, res){
res.render('topics/index.ejs',{ topics : allTopics })
});
Is the above code correct for route?
Also I have index.ejs file where I want to list all the topics (i.e. retrieve data from json response). How do I achieve this?
Your code as-is won't work but you could rewrite it as follows:
// notice how I am passing a callback rather than req/res
allTopics = function (callback) {
db.Topic.all({limit: 10}).success(function (topics) {
callback(topics);
});
};
// call allTopics and render inside the callback when allTopics()
// has finished. I renamed "allTopics" to "theData" in the callback
// just to make it clear one is the data one is the function.
app.get('/topics', function (req, res){
allTopics(function(theData) {
res.render('topics/index.ejs',{ topics : theData });
});
});
The following code is the user-facing part of a new node app we are building:
var loadInvoice = function(req, res, next) {
Invoice.findById(req.params.invoiceId, function (err, invoice) {
if (err) {
res.send(404, 'Page not found');
} else {
req.invoice = invoice;
next();
}
});
};
app.namespace('/invoices/:invoiceId', loadInvoice, function () {
app.get('', function(req, res){
var templateVals = {
//some template data
};
res.render('paymentselection', templateVals);
});
app.post('', function(req, res){
var data = {
// some data for the apiCall
};
someAPI.someRequest(data, function(err, data) {
console.log(res.status());
res.redirect(data.url);
});
});
});
The first method returns a confirmation page where the user presses a button to post to the same url, which triggers a redirect to an external website.
This all works exactly once. Every second request will crash the app with the message Cant set headers after they are sent. After carefull inspection of the code I could find no reason for this to happen so I added the console.log line which indeed confirms the location header has been set. But it is set to the value i got from someAPI on the previous request not the current one.
This makes absolutely no sense to me. I do not store this value anywhere nor do I do caching or persistence of this data in any way.
Does anybody know what could be causing this?
I use express, express-namespace, mogoose and swig
I found out the problem was being caused bij the 'Restler' libaray used within 'someAPI'. I have no idea how this is possible but swapping it out with something else fixed the problem.