I have an Express server running for the backend of a website with Sentry (v5.15.5) successfully implemented. I'm now trying to improve the error handling on the backend, as at the moment if something goes wrong with the request, the request is not ended and the client sits there waiting for a response, which it never gets.
To tell the client if the request has failed I'm using the custom error handler in the documentation:
app.use(function onError(err, req, res, next) {
// The error id is attached to `res.sentry` to be returned
// and optionally displayed to the user for support.
res.statusCode = 500;
res.end(res.sentry + "\n");
});
However, when I use this on my server, only the custom error handler runs - Sentry never creates an event for the error, but if I just use another custom error function, they both get called fine, which makes me think this is a Sentry issue. Here's the relevant parts of the server code:
...
const Sentry = require('#sentry/node');
...
const app = express()
Sentry.init({ dsn: process.env.SENTRY });
...
// Middlewares
app.use(Sentry.Handlers.requestHandler());
app.use(express.json())
app.use(helmet())
app.use(cors())
app.use(morgan('tiny'))
const controllers = require('./controllers')
const wrap = fn => (...args) => Promise
.resolve(fn(...args))
.catch(args[2])
// Routes
...
app.post('/test', authUser, wrap(controllers.testController))
...
app.use(Sentry.Handlers.errorHandler());
app.use(function onError(err, req, res, next) {
res.statusCode = 500
res.end(res.sentry + "\n")
})
app.listen(PORT, () => console.log(`APP RUNNING ON PORT ${PORT}`))
The controllers on the server make database requests, etc. so they are async functions - that's why I use the wrap function to catch promise rejections and pass them to the error handler. If I unwrap the controller then Sentry works fine, but then the server never sends the error to the client.
I expect I'm probably going about this wrong as it should be pretty simple to do, but no matter what I do I cannot get Sentry + async controllers + custom error handler to work. Any help would be appreciated.
(This may be an Express issue, if so let me know and I'll take it over there)
Thanks
For some reason Sentry's 'Filter out localhost' option (which was turned off but somehow got toggled on) doesn't actually filter out all local errors. When I removed the custom error handler and wrap function, the errors managed to get past the localhost filter. After I turned it back off all the errors came through on Sentry fine.
I want to have a Node web server serving pages and also set as an endpoint listening to webhooks. The example for the first comes from Rocket Rides, with the relevant code being:
const express = require('express');
// ...
const app = express();
// ...
// CRUD routes for the pilot signup and dashboard
app.use('/pilots', require('./routes/pilots/pilots'));
app.use('/pilots/stripe', require('./routes/pilots/stripe'));
// ...
// Index page for Rocket Rides
app.get('/', (req, res) => {
res.render('index');
});
// ...
// Start the server on the correct port
const server = app.listen(process.env.PORT || config.port, () => {
console.log('🚀 Rocket Rides server started:', config.publicDomain);
});
For the second, I use this tutorial with the following relevant code:
// Match the raw body to content type application/json
app.post('/webhook', bodyParser.raw({type: 'application/json'}), (request, response) => {
console.log("called!");
let event;
try {
event = JSON.parse(request.body);
} catch (err) {
response.status(400).send(`Webhook Error: ${err.message}`);
}
// Handle the event
switch (event.type) {
case 'payment_intent.succeeded':
const paymentIntentSucceeded = event.data.object;
break;
case 'payment_method.attached':
const paymentMethod = event.data.object;
break;
// ... handle other event types
default:
// Unexpected event type
return response.status(400).end();
}
// Return a response to acknowledge receipt of the event
response.json({received: true});
});
app.listen(8000, () => console.log('Webhooks running on port 8000'));
With both parts, the server does not handle the webhook request:
Webhooks running on port 8000
POST /webhook 404 590.525 ms - 1415
and the sender receives a 404.
When I comment out most of the code in the first part, the webhook request is handled properly:
Webhooks running on port 8000
called!
and the sender receives a 200.
I believe one of the routes from the web server is masking the route for the endpoint. I tried looking for one with this thread:
app._router.stack.forEach(function(middleware){
if(middleware.route){ // routes registered directly on the app
routes.push(middleware.route);
} else if(middleware.name === 'router'){ // router middleware
middleware.handle.stack.forEach(function(handler){
route = handler.route;
route && routes.push(route);
});
}
});
console.log(routes);
and the only relevant one was GET /.
If I include the code for the endpoint before the code for the router, the webhook is handled properly.
How can I find which route is masking the webhook endpoint?
Put the more specific route definitions first like this:
app.use('/pilots/stripe', require('./routes/pilots/stripe'));
app.use('/pilots', require('./routes/pilots/pilots'));
And, the more general route definitions later. That makes sure the more specific routes aren't gobbled up by the more general handlers.
Keep in mind that with app.use(), something like app.use('/pilots') will match any route that starts with /pilots which would include all your /pilots/stripe routes. So, you want to make sure and put the app.use('/pilots/stripe', ...) before the app.use('/pilots', ...).
Another observation. In your /webhook handler, you need to return after you send an error status so the rest of your request handler doesn't continue to run.
// Match the raw body to content type application/json
app.post('/webhook', bodyParser.raw({type: 'application/json'}), (request, response) => {
console.log("called!");
let event;
try {
event = JSON.parse(request.body);
} catch (err) {
response.status(400).send(`Webhook Error: ${err.message}`);
return; // <===== Add this
}
....
}
This appears to be a bug in the actual stripe documentation.
If I include the code for the endpoint before the code for the router, the webhook is handled properly.
I would guess that you have bodyparser middleware elsewhere in your server. If that middleware is BEFORE this route, then this route won't get to use its bodyParser.raw() and get the data the way it wants and it will not work properly. This is because whichever bodyParser middleware runs first reads the body and parses it and puts it wherever that middleware is configured to put it. Once the body is read, it's gone from the stream so any other middleware that comes along and also tries to read the body data from the stream will find the stream empty.
So, this route just has to be before any other body parsing middleware that might handle JSON.
If you provided a link to your full code, we could take a look and see where this is happening.
I am using postman to test a rest API I'm building for a project. I'm trying to send some data to a post method, but the data is getting lost somewhere between Postman and the endpoint.
I've tried tracing the data with console logs, but nothing comes out (req.body is undefined). I'm pretty sure the issue isn't with the endpoint or router, as the same error comes up in postman as in the console of my IDE, which means there's some sort of communication.
// json I'm putting into postman. validated with Jsonlint.com
{
"Name": "testN",
"file": "file1",
"Path": "/home/userf",
"userName": "user1"
}
// profileWrite.js
const dbProfileWrite = require('../...db-ProfileWrite');
const bodyParser = require('body-parser');
var app = express();
app.use(bodyParser.json());
// my post method
async function post(req, res, next) {
try {
console.log("attempting to parse data: " + req.body);
let profile = req.body;
console.log("data parsed, writing profiles");
profile= await dbProfileWrite.writeProfile(profile);
res.status(201).json(profile);
} catch (err) {
next(err);
}
}
module.exports.post = post;
UPDATE 7/15/19:I have recreated a microversion of my project that is having this issue on stackblitz. there's some sort of package error that I'm working on, but here's the link. I've recreated the methodology I'm using in my project with the router and all and looked at the example in the express docs. hopefully this helps isolate the issue.The data still comes in undefined when I post to this api through postman, so helpfully this much smaller view of the whole project helps.
Assuming you are using Express framework, by the look of the post function. You need to use a middlewear function to process request body using body-parser. Make sure you are using the correct parser in this case
app.use(bodyParser.json())
You don't need body-parser anymore it was put back in to the core of express in the form of express.json, simply use app.use(express.json()).
To access the body of your request use req.body, it should come with a object with the keys representing the json sent;
var app = express();
app.use(express.json());
async function post(req, res, next) {
try {
console.log("attempting to parse data: " + req.body);
let profile = req.body; // no need to use any functions to parse it
console.log("data parsed, writing profiles");
profile= await dbProfileWrite.writeProfile(profile);
res.status(201).json(profile);
console.log("profilecreated");
} catch (err) {
next(err);
}
}
See the express documentation
Solved the issue myself with a little help from John Schmitz. The issue was that I was defining the router and the server before actually telling it how to handle json bodies/ objects, so the body came through as the default undefined. In my index.js, the following is what fixed the code:
app.use(express.urlencoded({ extended: true }));
app.use(express.json());
app.use('/api/v1', router);
the key to this is that the app is told to use json and express.urlencoded before the router is declared. these actions have to happen in this order, and all before app.listen is called. once the app is listening, all of its params are set and you can't change them. Tl;dr: node is VERY picky, and you HAVE to define these things in the right place. thanks all for the help.
After spending close to a week trying to implement passport strategy for google token unsuccessfully, I decided write it myself using the Nodejs client library for Google API. I'm passing the accesstoken from angularjs in a $http request and want to return the logged in user to the call after authenticating on the server side.
I'm having trouble in passing the signed in user to the req object so I can pass it to the next middleware. Any help will be greatly appreciated.
**express code**
router
var gCtrl = require('./googleController.js');
app.post('/auth/google/accesstoken',
function (req, res) {
console.log('Im in the google auth middleware');
//below line is not returning the gUser so I can pass it to the next middleware
var gUser=gCtrl(req.token);
res.send(gUser);
},
function(req,res){
//do something with authenticated user
}
);
gCtrl Module that makes the Google API call
var ubCust = require('../models/ubCust.js');
var bodyParser = require('body-parser');
var google=require('googleapis');
var plus = google.plus('v1');
var OAuth2 = google.auth.OAuth2;
var oauth2client = new OAuth2('1094664898379-8u0muh9eme8nnvp95dafuc3rvigu4j9u.apps.googleusercontent.com','KQ_vNDaZzTvXDdsfgp5jqeZW','http://localhost:3000/auth/google/callback');
module.exports = function(acc_token){
//console.log(acc_token);
oauth2client.setCredentials({
access_token : acc_token
//refresh_token: token.refresh_token
});
plus.people.get({
userId:'me',
auth:oauth2client
},function(err,response){
if (err)
throw err;
//console.log('google user',response);
return response;
});
};
You are missing an understanding of how one returns values from an asynchronous operation. Where you have return response in your gCtrl module is just returning the value internally to whatever the async operation is inside of plus.people.get(). That will not return the value from your module.exports function. That function has long since returned.
You can read about the general concept of returning values from an async operation here: How do I return the response from an asynchronous call?. You will need to use either a callback function or a promise to communicate back an asynchronous value. You cannot return it directly.
Now, in your specific case, you could use your gCtrl module as Express middleware where you set the intermediate token value into the req object in your middleware handler and then use next() to communicate that it's now time to call your main handler for the request. You can do that like this:
// gtrl - Express middleware handler
// the purpose of this middleware is to process req.token
// and set req.tokenResponse for further processing
// in the next step of the middleware/request handler processing
module.exports = function(req, res, next) {
oauth2client.setCredentials({
access_token : req.token
//refresh_token: token.refresh_token
});
plus.people.get({
userId:'me',
auth:oauth2client
}, function(err,response){
if (err) {
// communicate back error in some fashion, may want to return a 5xx response for an internal error
next(err);
} else {
// set value into req object for later consumption
req.tokenResponse = response;
// call next handler in the chain
next();
}
});
};
// router
var gCtrl = require('./googleController.js');
// use gCtrl middleware to do async processing on req.token
app.post('/auth/google/accesstoken', gCtrl, function(req, res) {
// req.tokenResponse is set here by the gCtrl middleware
// do something with authenticated user
// send some sort of response here
res.send(...);
});
Note: to make this work, we have to specifically make the gCtrl exported function match the function signature of express middleware and it retrieves the token directly from the req object. You could make the gCtrl module independent of Express and just make it an async function that uses a callback or a promise to communicate back when it has it's data and then code the express handler to call that function and use that async return value appropriately. In my code above, I chose to use the already built middleware architecture for handling the async response.
Almost every Express app I see has an app.use statement for middleware but I haven't found a clear, concise explanation of what middleware actually is and what the app.use statement is doing. Even the express docs themselves are a bit vague on this. Can you explain these concepts for me please?
middleware
I'm halfway through separating the concept of middleware in a new project.
Middleware allows you to define a stack of actions that you should flow through. Express servers themselves are a stack of middlewares.
// express
var app = express();
// middleware
var stack = middleware();
Then you can add layers to the middleware stack by calling .use
// express
app.use(express.static(..));
// middleware
stack.use(function(data, next) {
next();
});
A layer in the middleware stack is a function, which takes n parameters (2 for express, req & res) and a next function.
Middleware expects the layer to do some computation, augment the parameters and then call next.
A stack doesn't do anything unless you handle it. Express will handle the stack every time an incoming HTTP request is caught on the server. With middleware you handle the stack manually.
// express, you need to do nothing
// middleware
stack.handle(someData);
A more complete example :
var middleware = require("../src/middleware.js");
var stack = middleware(function(data, next) {
data.foo = data.data*2;
next();
}, function(data, next) {
setTimeout(function() {
data.async = true;
next();
}, 100)
}, function(data) {
console.log(data);
});
stack.handle({
"data": 42
})
In express terms you just define a stack of operations you want express to handle for every incoming HTTP request.
In terms of express (rather than connect) you have global middleware and route specific middleware. This means you can attach a middleware stack to every incoming HTTP requests or only attach it to HTTP requests that interact with a certain route.
Advanced examples of express & middleware :
// middleware
var stack = middleware(function(req, res, next) {
users.getAll(function(err, users) {
if (err) next(err);
req.users = users;
next();
});
}, function(req, res, next) {
posts.getAll(function(err, posts) {
if (err) next(err);
req.posts = posts;
next();
})
}, function(req, res, next) {
req.posts.forEach(function(post) {
post.user = req.users[post.userId];
});
res.render("blog/posts", {
"posts": req.posts
});
});
var app = express.createServer();
app.get("/posts", function(req, res) {
stack.handle(req, res);
});
// express
var app = express.createServer();
app.get("/posts", [
function(req, res, next) {
users.getAll(function(err, users) {
if (err) next(err);
req.users = users;
next();
});
}, function(req, res, next) {
posts.getAll(function(err, posts) {
if (err) next(err);
req.posts = posts;
next();
})
}, function(req, res, next) {
req.posts.forEach(function(post) {
post.user = req.users[post.userId];
});
res.render("blog/posts", {
"posts": req.posts
});
}
], function(req, res) {
stack.handle(req, res);
});
After simplifying things, a web server can be seen as a function that takes in a request and outputs a response. So if you view a web server as a function, you could organize it into several pieces and separate them into smaller functions so that the composition of them will be the original function.
Middlewares are the smaller functions that you can compose with others and the obvious benefit is that you can reuse them.
I add a late answer to add something not mentioned in the previous answers.
By now it should be clear that middleware is/are function(s) run between the client request and the server answer. The most common middleware functionality needed are error managing, database interaction, getting info from static files or other resources. To move on the middleware stack the next callback must be called, you can see it in the end of middleware function to move to the next step in the flow.
You can use the app.use approach and have a flow like this:
var express = require('express'),
app = express.createServer(),
port = 1337;
function middleHandler(req, res, next) {
console.log("execute middle ware");
next();
}
app.use(function (req, res, next) {
console.log("first middle ware");
next();
});
app.use(function (req, res, next) {
console.log("second middle ware");
next();
});
app.get('/', middleHandler, function (req, res) {
console.log("end middleware function");
res.send("page render finished");
});
app.listen(port);
console.log('start server');
but you can also use another approach and pass each middleware as function arguments. Here is a example from the MooTools Nodejs website where midleware gets the Twitter, Github and Blog flow before the response is sent back to the client. Note how the functions are passed as arguments in app.get('/', githubEvents, twitter, getLatestBlog, function(req, res){. Using app.get will only be called for GET requests, app.use will be called for all requests.
// github, twitter & blog feeds
var githubEvents = require('./middleware/githubEvents')({
org: 'mootools'
});
var twitter = require('./middleware/twitter')();
var blogData = require('./blog/data');
function getLatestBlog(req, res, next){
blogData.get(function(err, blog) {
if (err) next(err);
res.locals.lastBlogPost = blog.posts[0];
next();
});
}
// home
app.get('/', githubEvents, twitter, getLatestBlog, function(req, res){
res.render('index', {
title: 'MooTools',
site: 'mootools',
lastBlogPost: res.locals.lastBlogPost,
tweetFeed: res.locals.twitter
});
});
expressjs guide has pretty neat answer to your question, I highly recommend you to read that, I am posting a short snippet of the guide, the guide is quite good.
Writing middleware for use in Express apps
Overview
Middleware functions are functions that have access to the request object (req), the response object (res), and the next function in the application’s request-response cycle. The next function is a function in the Express router which, when invoked, executes the middleware succeeding the current middleware.
Middleware functions can perform the following tasks:
Execute any code.
Make changes to the request and the response objects.
End the request-response cycle.
Call the next middleware in the stack.
If the current middleware function does not end the request-response cycle, it must call next() to pass control to the next middleware function. Otherwise, the request will be left hanging.
Example
Here is an example of a simple “Hello World” Express application. The remainder of this article will define and add two middleware functions to the application: one called myLogger that prints a simple log message and another called requestTime1 that displays the timestamp of the HTTP request.
var express = require('express')
var app = express()
app.get('/', function (req, res) {
res.send('Hello World!')
})
app.listen(3000)
Middleware function myLogger
Here is a simple example of a middleware function called “myLogger”. This function just prints “LOGGED” when a request to the app passes through it. The middleware function is assigned to a variable named myLogger.
var myLogger = function (req, res, next) {
console.log('LOGGED')
next()
}
Notice the call above to next(). Calling this function invokes the next middleware function in the app. The next() function is not a part of the Node.js or Express API, but is the third argument that is passed to the middleware function. The next() function could be named anything, but by convention it is always named “next”. To avoid confusion, always use this convention.
To load the middleware function, call app.use(), specifying the middleware function. For example, the following code loads the myLogger middleware function before the route to the root path (/).
var express = require('express')
var app = express()
var myLogger = function (req, res, next) {
console.log('LOGGED')
next()
}
app.use(myLogger)
app.get('/', function (req, res) {
res.send('Hello World!')
})
app.listen(3000)
Every time the app receives a request, it prints the message “LOGGED” to the terminal.
The order of middleware loading is important: middleware functions that are loaded first are also executed first.
If myLogger is loaded after the route to the root path, the request never reaches it and the app doesn’t print “LOGGED”, because the route handler of the root path terminates the request-response cycle.
The middleware function myLogger simply prints a message, then passes on the request to the next middleware function in the stack by calling the next() function.
This post will only contain myLogger middleware, for further post you could go to the original expressjs guide here.
=====Very very simple explanation=====
Middlewares are often used in the context of Express.js framework and are a fundamental concept for node.js . In a nutshell, Its basically a function that has access to the request and response objects of your application. The way I'd like to think about it, is a series of 'checks/pre-screens' that the request goes through before the it is handled by the application. For e.g, Middlewares would be a good fit to determine if the request is authenticated before it proceeds to the application and return the login page if the request is not authenticated or for logging each request. A lot of third-party middlewares are available that enables a variety of functionality.
Simple Middleware example:
var app = express();
app.use(function(req,res,next)){
console.log("Request URL - "req.url);
next();
}
The above code would be executed for each request that comes in and would log the request url, the next() method essentially allows the program to continue. If the next() function is not invoked, the program would not proceed further and would halt at the execution of the middleware.
A couple of Middleware Gotchas:
The order of middlewares in your application matters, as the request would go through each one in a sequential order.
Forgetting to call the next() method in your middleware function can halt the processing of your request.
Any change the req and res objects in the middleware function, would make the change available to other parts of the application that uses req and res
Middlewares are functions executed in the middle after the input/source then produces an output which could be the final output or could be used by the next middleware until the cycle is complete.
It is like a product that goes through an assembly line where it gets modified as it moves along until it gets completed, evaluated or gets rejected.
A middleware expects some value to work on (i.e. parameter values) and based on some logic the middleware will call or not call the next middleware or send a response back to the client.
If you can't still grasp the middleware concept, it is in a way similar to the Decorator or Chain of command patterns.
Middleware is a subset of chained functions called by the Express js routing layer before the user-defined handler is invoked. Middleware functions have full access to the request and response objects and can modify either of them.
The middleware chain is always called in the exact order in which it has been defined, so it is vital for you to know exactly what a specific piece of middleware is doing. Once a middleware function finishes, it calls the next function in the chain by invoking its next argument as function. After the complete chain gets executed,the user request handler is called.
Keep things simple, man!
Note: the answer is related to the ExpressJS builtin middlware cases, however there are different definitions and use cases of middlewares.
From my point of view, middleware acts as utility or helper functions but its activation and use is fully optional by using the app.use('path', /* define or use builtin middleware */) which don't wants from us to write some code for doing very common tasks which are needed for each HTTP request of our client like processing cookies, CSRF tokens and ..., which are very common in most applications so middleware can help us do these all for each HTTP request of our client in some stack, sequence or order of operations then provide the result of the process as a single unit of client request.
Example:
Accepting clients requests and providing back responses to them according to their requests is the nature of web server technology.
Imagine if we are providing a response with just "Hello, world!" text for a GET HTTP request to our webserver's root URI is very simple scenario and don't needs anything else, but instead if we are checking the currently logged-in user and then responding with "Hello, Username!" needs something more than usual in this case we need a middleware to process all the client request metadata and provide us the identification info grabbed from the client request then according to that info we can uniquely identify our current user and it is possible to response to him/her with some related data.
Hope it to help someone!
In very basic term if i want to explain it like this i learn this from traversymedia youtube channel express crash course.
ok so middle ware is a function who execute after you make a call to your route like this.
var logger = function(req, res, next){
console.log('logging...');
next();
}
app.use(logger);
This logger function execute every time you refresh your page that means you can write anything in it that you required to do after your page get rendered any operation api call, reset things basically anything. and put this middleware before your route function order of middleware is really important or it dons't work