Is it possible to trace Node.js applications the same way it's possible to trace applications that use threads by assigning a request id and using it in log output? There are third-party tools that can do it (NewRelic, Rollbar), but how do they work? How does one implement a similar functionality? I can do something like this
function foo(x, logger) {
logger.info("Running foo")
}
function bar(a, b, logger) {
logger.info(`Running bar with ${a} and ${b}`)
}
app.get('/', (req, res) => {
const requestId = uuid()
const logger = {
info: (msg) => console.log(`${requestId} ${msg}`)
}
foo(1, logger)
bar(2, 3, logger)
}
But that requires explicitly passing logger object. What I want to do is to something like this in downstream functions
function foo(x) {
const logger = require('logger')
logger.info('Running foo') // => "[ce089d84-3679-42ae-b05c-2d0a071062f0] Running foo"
}
You can do it with domain and express for example. You can test this script, just save it in a file run npm i express uniqid then node index.js and open browser at localhost:8080/test, just reload it and you will see the id changing.
const app = require('express')();
const uniqid = require('uniqid');
app.use((req, res, next) => {
const nDomain = require('domain').create();
nDomain.id = uniqid();
nDomain.add(req);
nDomain.add(res);
nDomain.run(next);
nDomain.on('error', next);
});
app.get('/test', function(req, res) {
console.log(process.domain.id, 'id');
});
const server = app.listen(8080);
This way, just create a logger in a package which uses process.domain.id when logging. We use something similar at work to track which request triggered a log.
Beware, domain is pending deprecation. Please read the doc https://nodejs.org/api/domain.html
Related
I'm running a standard NodeJs 8 with Express and currently when a request for an existing path but un-supported method comes in, Express return 404.
For example 'POST /login' is supported, but 'GET /login' is not, but it returns 404.
How can I make Express return 405 in such a case?
Here's the routes file:
const express = require('express');
const router = express.Router();
const loginController = require('../controllers/login');
router.route('/login').post(loginController.loginUser);
module.exports = router;
Please advise.
You can simply add the .all() handler to your route chain, like so:
const methodNotAllowed = (req, res, next) => res.status(405).send();
router
.route(`/login`)
.post(loginController.loginUser)
.all(methodNotAllowed);
Explanation
This works because requests are passed to the handlers in the order they are attached to the route (the request "waterfall"). The .post() handler will catch your POST requests, and the rest will fall through to the .all() handler.
Also see this question for more details.
Authenticating all POST routes
If you would like to ensure that the user is logged in for all POST requests, but return a 405 response for any other requests, you can use a regular expression to match all routes with router.post('*'), like so:
router
.post(`*`, loginController.loginUser)
.all(methodNotAllowed);
The problem with this approach, however, is that no 404 errors will ever be returned to the client, only 405. Therefore I recommend attaching the methodNotAllowed handler to each individual route, like in the first code snippet above. This approach will return 404 errors for routes that don't exist, but 405 errors for routes that do.
Determining the available methods for a route
To determine which methods are allowed for a route, use router.stack:
app.use((req, res, next) => {
const methods = router.stack
// Filter for the route that matches the currently matched route
.filter(layer => layer.route.path === req.path)[0]
.route
.methods;
if (!methods[req.method]) methodNotAllowed(req, res, next);
else next();
});
You can try this that way:
app.route("/login")
.get((req, res) => {
/* HANDLE GET */
})
.post((req, res) => {
/* HANDLE POST */
})
.all((req, res) => {
res.status(405).send();
});
How it works?
If request matches the route. It will go through the handlers. If a handler is present, it will be handled using that specific one. Otherwise, it will reach the 'all' handler that will set the status code to 405 and send the response.
Here You can find the discussion about it:
405 issue
#You question below:
You can try that way:
loginRoutes.js content:
const router = require('express').Router();
router.route('/')
.get((req, res) => {
res.status(200).send()
})
module.exports = router
server file content:
const express = require('express')
const app = express();
const router = express.Router();
const loginRoutes = require('./loginRoutes')
const PORT = process.env.PORT || 8080;
router.use('/login', loginRoutes)
router.route('/login').all((req, res) => { res.status(405).send() })
app.use(router);
app.listen(PORT, () => console.log(`started on port: ${PORT}`))
You can use this snippet of code to automatically send 405 status code when route from the same path exist but not with the current method
app.use(function (req, res, next) {
const AllLayers = app._router.stack
const Layers = AllLayers.filter(x => x.name === 'bound dispatch' && x.regexp.test(req.path))
const Methods = [];
Layers.forEach(layer => {
for (let method in layer.route.methods) {
if (layer.route.methods[method] === true) {
Methods.push(method.toUpperCase());
}
}
})
if (Layers.length !== 0 && !Methods.includes(req.method)) {
res.setHeader('Allow', Methods.join(','))
if (req.method === "OPTIONS") {
return res.send(Methods.join(', '))
}
else {
return res.sendStatus(405);
}
}
else {
next();
}
});
Hope this could be helpfull to someone
If you want to determine what methods COULD have been used you need to do a lot of digging in the app function you start your server with, and through some string manipulation and the like you can figure out what the possible methods are and return them in the error. If you're interested in how its done check out https://github.com/Justinlkirk/express-ez-405 or just use the npm package here https://www.npmjs.com/package/express-ez-405
I am trying to build an express app and I need to create some singletons (like db object in Sequelizer).
app.js
app.use(...);
app.use(...);
var serviceLocator = {
foo: require('foo'),
bar: require('bar')
}; //object holding singletons.
app.use('/api/todo', new todoRoutes(serviceLocator));
todoRoutes.js
module.exports = (serviceLocator) => {
var router = express.Router();
router.get('/', (req,res,next) => {
//use serviceLocator.foo
});
router.get('/:id',(req,res,next) => {
//use serviceLocator.bar
});
};
Is this a good practice?
(I've also read about building singletons using require caching, but I have concerns since in the official docs they say that require "may not" return the same object).
How I usually do it looks something like this:
app.js
const db = require('./path/to/db/singleton');
const sql = require('./path/to/Sequelizer/singleton');
app.use(...);
app.use((req, res, next) => {
req.db = db;
next();
});
app.use((req, res, next) => {
req.sql = sql;
next();
});
app.use('/api/todo', require('./todoRoutes');
todoRoutes.js
const express = require('express');
const router = express.Router();
router.get('/', (req,res,next) => {
console.log(req.db);
});
router.get('/:id',(req,res,next) => {
console.log(req.sql);
});
module.exports = router;
The overall is just to add some middleware that adds it to the req, which is going to go through the pipeline for you. You could namespace it too by adding it to req.server.<thing> or something.
You can also do this to generate a request ID by bringing in uuid and attaching an ID to every req in another middleware that you can use in your log statements later, that way you can track requests through.
Alternatively, just require those singletons into the todoRoutes.js and use them directly. They're singletons on your server, so I don't see any issue with that.
I'm developing an app with node JS, the app generates a report calling the endpoint api.example.com/generate-report
But this report takes around 1 minute on be generated, then I want to implement something like this:
User click on generate report
System return response {generating:"ok"}
After the system generate the report send a notification (this I what I know how to do)
User get the report
Is this possible with nodejs?
After I do some research, this can be easily done using Promises.
To run the following code it's necessary to install express and node uuid
npm install --save express
npm install --save uuid
node index.js
The source code of index is:
//index.js
const express = require("express");
const app = express();
const PORT = process.env.PORT || 5000;
const uuidV1 = require('uuid/v1');
// this is where we'll store the results of our jobs by uuid once they're done
const JOBS = {};
app.get("/", (req, res) => {
res.send("It works!");
});
app.get("/startjob", (req, res) => {
let times = [100, 1000, 10000, 20000];
let promises = [];
for (let time of times) {
promises.push(new Promise((resolve, reject) => {
setTimeout(resolve, time, `${time} is done.`);
}));
}
// obviously, you'd want to generate a real uuid here to avoid collisions
let uuid = uuidV1();
console.log(uuid);
Promise.all(promises).then(values => { JOBS[uuid] = values; });
res.redirect(`progress/${uuid}`);
});
app.get("/progress/:uuid", (req, res) => {
if (JOBS[req.params.uuid] === undefined) {
res.send("Still processing your request.");
} else {
res.send(`Here's your result: ${JOBS[req.params.uuid]}.`);
// instead of immediately deleting the result of the job (and making it impossible for the user
// to fetch it a second time if they e.g. accidentally cancel the download), it would be better
// to run a periodic cleanup task on `JOBS`
delete JOBS[req.params.uuid];
}
});
app.listen(PORT, () => {
console.log(`Listening on localhost:${PORT}.`);
});
When the code runs you will be redirected to /process/uuid and I get the status of the process.
This needs some improvements because I want the response like "{process:uuid}" and I can store this on my Local Storage to use after.
Well, I hope this help to someone.
librarys like express-validation and passport add methods to the req-variable, like req.assert and req.user. I know I can do this from inside a route:
app.get('/', (req,res,next) => {
req.foo = bar;
next();
}
But how can I do it from a library or external module?
In short: You expose a handler function which your consumers will need to app.use().
Here's the general approach. Consider the following module:
module.exports = function myExtension() {
return function myExtensionHandler(req, res, next) {
// This is called for every request - you can extend req or res here
req.myFun = myFun
// Make sure to call next() in order to continue the
// chain of handlers
return next()
}
}
function myFun() {
// My magic function
}
Now, from a consumer perspective:
const express = require('express')
const myExtension = require('my-extension')
const app = express()
// Here you tell Express to use your extension for incoming requests
app.use(myExtension())
// ...
Is it possible to define a unique request Id that is included in each log statement without handing the logger to each method/function call?
Technologies in use: NodeJS, Express, Winston
Edited
Finally, I have created a library that makes all the work.
https://github.com/davicente/express-logger-unique-req-id
It is a wrapper of Winston library, so you can use it the same way.
Let me know if it helps you
We had this same problem in several projects, and I couldn't finde any complete solution for this question. We are using same technologies (Node.js, Express.js and Winston for logs)
I found a solution to this using a couple of libraries and wrapping Winston library:
- node-uuid for creating unique identificators for each request
- continuation-local-storage for sharing this IDs among different modules without sending req object in all the calls.
First I need to create and set the unique identificator with each request. I do it in the middleware:
var uuid = require('node-uuid');
var createNamespace = require('continuation-local-storage').createNamespace;
var myRequest = createNamespace('my request');
// Run the context for each request. Assign a unique identifier to each request
app.use(function(req, res, next) {
myRequest.run(function() {
myRequest.set('reqId', uuid.v1());
next();
});
});
After that I had to wrap Winston library, recovering the id from the context and adding to the message of the log:
var winston = require('winston');
var getNamespace = require('continuation-local-storage').getNamespace;
// Wrap Winston logger to print reqId in each log
var formatMessage = function(message) {
var myRequest = getNamespace('my request');
message = myRequest && myRequest.get('reqId') ? message + " reqId: " + myRequest.get('reqId') : message;
return message;
};
var logger = {
log: function(level, message) {
winstonLogger.log(level, formatMessage(message));
},
error: function(message) {
winstonLogger.error(formatMessage(message));
},
warn: function(message) {
winstonLogger.warn(formatMessage(message));
},
verbose: function(message) {
winstonLogger.verbose(formatMessage(message));
},
info: function(message) {
winstonLogger.info(formatMessage(message));
},
debug: function(message) {
winstonLogger.debug(formatMessage(message));
},
silly: function(message) {
winstonLogger.silly(formatMessage(message));
}
};
module.exports = logger;
I think it was a little bit complex, so I decided to write it down in a post. You can get more information from there: Express.js: Logging info with global unique request ID – Node.js
I hope this helps with your problem.
This answer has a problem: the counter goes back to 0 every time the node process is restarted. Turns out there is fairly simple to work around. You simply add an express middleware that tags each request called with a UUID using the uuid package.
For uuid Pre-2.0
const uuid = require('uuid');
const app = express();
app.use(function (req, next) {
req.id = uuid.v4();
next();
});
For uuid 3.0+
const uuidv4 = require('uuid/v4');
const app = express();
app.use(function (req, next) {
req.id = uuidv4();
next();
});
At the very beginning of your request handling add something like the following (or put it in its own file):
var makeID = (function() {
var index = 0;
return function() {
return index++;
}
})();
app.use(function(req, res, next) {
req.id = makeID()
next()
})
This will give every request a unique (sequential) id. Do not allow people to identify themselves with it, only use it internally!
When logging in Winston, you can enter metadata, to be displayed after the message (in the form ${name}=${value}), which looks like this
app.use(function(req, res) {
winston.log('info', 'Test Log Message', { id: req.id });
res.end("Done.")
});
Hope this is helpful.