I am writing and API in express.js. the original API I wrote only utilized routes and raw SQL queries. I have since rewritten the API for the most part NOW using an ORM to react models and migrations.
My question what is the difference and use cases for middleware and controllers. currently only using middleware because most sources online online only explain what a middleware is.
I don't understand the use case of a controller. and I don't want to omit it from my API if its used in proper programming conventions
You should see middleware as a step in your API and controllers as the entity that will actually respond to the requests.
Bellow is an example where authenticationMiddleware is a middleware because it is a step during the processing but should not return the response. It can though, in case of error.
Then getItems actually handle the logic specific to this calls.
As a rule of thumb, middleware are often reused more than once and often they do not response. On contrary, controller respond and are most of the time specific to one endpoint.
const express = require("express");
const app = express();
function authenticationMiddleware(req, res, next) {
// Check that the user is authenticated using req.headers.Authorization
// for example
if (authenticated) {
// The user is authenticated, we can go to the next step
next();
} else {
// The user is not authenticated, we stop here
res.status(401);
res.send("Error during authentication");
}
}
function getItems(req, res, next) {
// Here we focus on the actual response, we assume that the user is authenticated
res.send({ items: [] });
}
app.get("/items", authenticationMiddleware, getItems);
app.post("/items", authenticationMiddleware, createItems); // Re-use the same middleware
app.listen(3000);
If you're referring to the node/express terminology, middleware are simply the callbacks used by the routing functions/methods (get, set, delete, use, etc). Callbacks can either send or not send responses back to the client. The callbacks are really the 'controllers' if you'd like to call them that (eg: ASP.NET Core MVC) but that's up to you. Below is a quote from Express. Note the term 'middleware' and where there's no mention of a 'controller'.
Middleware functions are functions that have access to the request object (req), the response object (res), and the next middleware
function in the application’s request-response cycle. The next
middleware function is commonly denoted by a variable named next.
Middleware functions can perform the following tasks:
Execute any code.
Make changes to the request and the response objects.
End the request-response cycle.
Call the next middleware function in the stack.
'Express' also defines different types of middleware which is useful:
Application-level middleware
Router-level middleware
Error-handling middleware
Built-in middleware Third-party middleware
Here's another nice look at it from Mozilla's pov that does mention a few controller/callback examples.
Beyond that, you can define what a 'controller' is within your team and the naming convention follows from there. Key is your SOLID profile and how you separate your concerns.
Related
My understanding is that a middleware function is a route handler with the exception that it may invoke the next function parameter to pass control on to the middleware function on the stack. Is this the only difference between a standard route handler and a middleware function?
Most of what you're talking about is semantics. In ExpressJS, middleware can be a route handler or a route handler can behave as middleware. So, there is not a hard and fast line between the two. But, when people refer to middleware or a route handler in a programming discussion, they usually mean something slightly different for each...
Middleware
As generic terms, middleware is code that examines an incoming request and prepares it for further processing by other handlers or short circuits the processing (like when it discovered the user is not authenticated yet). Some examples:
Session Management. Parse cookies, look for session cookie, lookup session state for that cookie and add session info to the request so that other handlers down the line have ready access to the session object without any additional work on their part. In express, this would be express.session().
Authentication. Check if the user is trying to access a portion of the site that requires authentication. If so, check if their authentication credentials are good. If not, send an error response and prevent further processing. If so, allow further processing.
Parsing of Cookies. Parse incoming cookies into an easy-to-use data structure a request handler can have easy access to cookie data without each having to parse them on their own. This type of middleware is built into Express and happens automatically.
Parsing and Reading of POST/PUT bodies. If the incoming request is a POST or PUT, the body of the request may contain data that is needed for processing the request and needs to be read from the incoming stream. Middleware can centralize this task reading the body, then parsing it according to the data type and putting the result into a known request parameter (in Express this would be req.body). Express has some ready-to-use middleware for this type of body parsing with express.json() or express.urlencoded(). A middleware library like multer is for handling file uploads.
Serve static files (HTML, CSS, JS, etc...). For some groups of URLs, all the server needs to do is to serve a static file (no custom content added to the file). This is common for CSS files, JS files and even some HTML files. Express provides middleware for this which is called express.static().
Route Handler
As a generic term, a route handler is code that is looking for a request to a specific incoming URL such as /login and often a specific HTTP verb such as POST and has specific code for handling that precise URL and verb. Some examples:
Serve a specific web page. Handle a browser request for a specific web page.
Handle a specific form post. For example, when the user logs into the site, a login for is submitted to the server. This would be handled by a request handler in Express such as app.post("/login", ...).
Respond to a specific API request. Suppose you had an API for a book selling web-site. You might provide in that API the ability to get info on a book by its ISBN number. So, you design an api that supports a query for a particular book such as /api/book/list/0143105426 where 0143105426 is the ISBN number for the book (a universal book identifier). In that case, you'd create a request handler in Express for a URL that looks like that: app.get('/api/book/list/:isbn', ...). The request handler in Express could then programmatically examine req.parms.isbn to get the request isbn number, look it up in the database and return the desired info on the book.
So, those are somewhat generic descriptions of middleware vs. request handlers in any web server system in any language.
In Express, there is no hard and fast distinction between the two. Someone would generally call something middleware that examines a bunch of different requests and usually prepares the request for further processing. Someone would generally call something a route handler that is targeted at a specific URL (or type of URL) and whose main purpose is to send a response back to the client for that URL.
But, the way you program Express, the distinction is pretty blurry. Express offers features for handling routes such as:
app.use()
app.get()
app.post()
app.put()
app.delete()
app.all()
Anyone of these can be used for either middleware or a route handler. Which would would call a given block of code has more to do with the general intent of the code than exactly which tools in Express it uses.
More typically, one would use app.use() for middleware and app.get() and app.post() for route handlers. But there are use cases for doing it differently than that as it really depends upon the particular situation and what you're trying to do.
You can even pass more than one handler to a given route definition where the first one is middleware and followed by a route handler.
app.get("/admin", verifyAuth, (req, res) => {
// process the /admin URL, auth is already verified
req.sendFile("...");
});
It is common for middleware to be active for a large number of different requests. For example, you might have an authentication middleware that prevents access to 95% of the site if the user isn't already logged in (say everything except the a few generally information pages such as the homepage and the login and account creation pages).
It is also common to have middleware that is active for all HTTP verbs such as GET, POST, DELETE, PUT, etc... In express, this would usually be app.use() or app.all(). Request handlers are usually only for one particular verb such as app.get() or app.post().
You might have session middleware that loads a session object (if one is available) for every single request on the site and then passes control on to other handlers that can, themselves, decide whether they need to access the session object or not.
It is common for request handlers to be targeted at a specific URL and only active for that specific URL. For example, the /login URL would typically have one request handler that renders that particular page or responds to login form requests.
Path Matching for app.use() is Different
In Express, there's one other subtle difference. Middleware is typically specified with:
app.use(path, handler);
And, a route is typically specified with:
app.get(path, handler);
app.post(path, handler);
app.put(path, handler);
// etc...
app.use() is slightly more greedy than app.get() and the others in how it matches the path. app.get() requires a full match. app.use() is OK with a partial match. Here are some examples:
So, for a URL request /category:
app.use("/category", ...) matches
app.get("/category", ...) matches
For a URL request /category/fiction:
app.use("/category", ...) matches
app.get("/category", ...) does not match
You can see that app.use() accepts a partial URL match, app.get() and it's other cousins do not accept a partial URL match.
Now, of course, you can use app.get() for middleware if you want and can use app.use() for request handlers if you want, but typically one would use app.use() for middleware and app.get() and its cousins for request handlers.
Ok, let's explore some definitions first.
Middleware function
definition from the express documentation:
Middleware functions are functions that have access to the request object (req), the response object (res), and the next function in the application’s request-response cycle. The next function is a function in the Express router which, when invoked, executes the middleware succeeding the current middleware.
Handler function or callback function
definition from express Documentation:
These routing methods specify a callback function (sometimes called “handler functions”) called when the application receives a request to the specified route (endpoint) and HTTP method. In other words, the application “listens” for requests that match the specified route(s) and method(s), and when it detects a match, it calls the specified callback function.
Here Routing methods are the methods derived from HTTP requests and the all() method that matches all the HTTP methods. But remind, the use() method is not a routing method. Let's go to express documentation for clarification.
Routing Methods
From express documentation:
A route method is derived from one of the HTTP methods and is attached to an instance of the express class. Express supports methods that correspond to all HTTP request methods: get, post, and so on. There is a special routing method, app.all(), used to load middleware functions at a path for all HTTP request methods.
So we see that middleware functions and handler functions are not opposite of each other.
Middleware functions are functions that take an extra argument, next along with req and res which is used to invoke the next middleware function.
app.use() and the other HTTP derived methods(app.get(), app.all() etc) can use middleware functions. The difference between handler function and middleware function is same on both app and router object.
On the other hand, a handler function is a function that is specified by the routing methods.
So when any of the routing methods pass a function that has req, res, and next then it is both a middleware function and a handler function.
But for app.use() if the function has req, res, and next, then it is only a middleware function.
Now, what about the functions which only have req and res don't have the next argument?!
They are not middleware functions by definition.
If such function is used on routing methods then they are only handler functions. We use such a handler function which is not a middleware when it is the only one callback function. Because in such cases there is no need for next which calls the next middleware function.
If they are used on app.use() then they are not middleware, nor handler function.
Ok, enough of the definitions, But is there any difference between middleware functions of app.use() and handler & middleware functions of routing methods??
They look similar since both have the next argument and works almost the same. But there is a subtle difference.
The next('route') works only on handler & middleware functions. So app.use() can not invoke next('route') since they can have only middleware functions.
According to express Documentation:
next('route') will work only in middleware functions that were loaded by using the app.METHOD() or router.METHOD() functions.
METHOD is the HTTP method of the request that the middleware function handles (such as GET, PUT, or POST) in lowercase.
If you know what is next('route') then the answer is finished here for you :).
In case you don't, you can come along.
next('route')
So let's see what is next('route').
From here we will use the keyword METHOD instead of get, post, all, etc.
From the previous middleware function definition, we see app.use() or app.METHOD() can take several middleware functions.
From the express documentation:
If the current middleware function does not end the request-response cycle, it must call next() to pass control to the next middleware function. Otherwise, the request will be left hanging.
We see each middleware functions have to either call the next middleware function or end the response.
But sometimes in some conditions, you may want to skip all the next middleware functions for the current route but also don't want to end the response right now. Because maybe there are other routes which should be matched. So to skip all the middleware functions of the current route without ending the response, you can run next('route'). It will skip all the callback functions of the current route and search to match the next routes.
For Example (From express documentation):
app.get('/user/:id', function (req, res, next) {
// if the user ID is 0, skip to the next route
if (req.params.id === '0') next('route')
// otherwise pass the control to the next middleware function in this stack
else next()
}, function (req, res, next) {
// send a regular response
res.send('regular')
})
// handler for the /user/:id path, which sends a special response
app.get('/user/:id', function (req, res, next) {
res.send('special')
})
See, here in a certain condition(req.params.id === '0') we want to skip the next callback function but also don't want to end the response because there is another route of the same path parameter which will be matched and that route will send a special response. (Yeah, it is valid to use the same path parameter for the same METHOD several times. In such cases, all the routes will be matched until the response ends). So in such cases, we run the next('route') and all the callback function of the current route is skipped. Here if the condition is not met then we call the next callback function.
This next('route') behavior is only possible in the app.METHOD() functions.
Recalling from express documentation:
next('route') will work only in middleware functions that were loaded by using the app.METHOD() or router.METHOD() functions.
That means this next('route') can be invoked only from handler & middleware functions. The only middleware functions of app.use() can not invoke it.
Since skipping all callback functions of the current route is not possible in app.use(), we should be careful here. We should only use the middleware functions in app.use() which need not be skipped in any condition. Because we either have to end the response or traverse all the callback functions from beginning to end, we can not skip them at all.
You may visit here for more information
I am overall clueless about how and why you set up a node.js app, and how any of the app.use functions work - the tutorials on it don't explain the why of anything.
Anyway, I have socket.io, res.locals and index.js set up like so in the app.js root file.
const sockets = require('./models/socket')(io)
app.use(function (req, res, next) {
res.locals.user_id = req.session.user_id;
next();
});
const routes = require('./routes/index');
app.use('/', routes);
I'd like to be able to access res.locals in the socket.js model, like I can in index.js found in the routes folder.
I can't guess how to go about doing this. If anybody is able to explain how and why I can or can't that would be a bonus. Thanks!
Welcome to Expressjs, there are a few fundamentals you should probably research before going any further, they'll help solve some of your confusion. I'll give a brief explanation of them but I suggest you do further research. I'll then answer your actual question at the end.
Middleware and app.use
Expressjs is built upon an idea that everything is just "middleware". Middleware is a function which runs as part of a request chain. A request chain is essentially a single client request, which then goes through a chain of a number of middleware functions until it either reaches the end of the chain, exits early by returning a response to the client, or errors.
Express middleware is a function which takes the following three arguments.
req (request) - Representing the request made by a client to your
server.
res (response) - Representing the response you will return to
the client.
next - A way of telling express that your current
middleware function is done, and it should now call the next piece of
middleware. This can either be called "empty" as next(); or with an
error next(new Error());. If it is called empty, it will trigger
the next piece of middleware, if it is called with an error then it
will call the first piece of error middleware. If next is not called at the
end of a piece of middleware, then the request is deemed finished and the
response object is sent to the user.
app.use is a way of setting middleware, this means it will run for every request (unless next() is either not called by the previous piece of middleware for some reason, or it's called with an error). This middleware will run for any HTTP request type (GET, POST, PUT, DELETE, etc).
app.use can take multiple arguments, the important ones for beginners to learn are: app.use(func) and app.use(path, func). The former sets "global" middleware which runs no matter what endpoint (url path) the client requests, the latter (with a specific path) is run only if that specific path is hit. I.e. app.use('/hello', (req, res, next) => { res.send('world'); }); will return "world" when the endpoint "/hello" is hit, but not if the client requests "/hi". Where as app.use((req, res, next) => { res.send('world'); }); would return "world" when you hit any endpoint.
There are more complex things you can do with this, but that's the basics of attaching middleware to your application. The order they are attached to the application, is the order in which they will run.
One more thing, this will blow your mind, an express application made with the standard const app = express() can also be used as middleware. This means you can create several express applications, and then mount them using app.use to a single express application. This is pretty advanced, but does allow you to do some really great things with Express.
Why can you not access res.locals in socket.io? (The real question)
Within your middleware handler, you are setting up a res.locals.use_id property. This only lives with that individual request, you can pass it around as long as the request is alive by passing it into other functions, but outside of that request it doesn't exist. res is literally the response object that tells Express how to respond to the clients request, you can set properties of it during the request but once that HTTP request has ended it's gone.
Socket.io is a way of handling web socket requests, not standard HTTP requests. Thus, in a standard express HTTP request you will not be able to hand off the connection to anything with socket.io, because the connection is a single short lived HTTP request. Likewise, you won't be able to do the same the other way.
If you wish to find the users id in a socket.io request, you'll have to do this within the socket.io request itself.
Right now, you're entering a piece of middleware for an Express.js request, you are then calling next() which runs the next piece of express middleware, at no point does it cross over into Socket.io realms. This is often confused by tutorials because Socket.io can handle requests across the same port as Express is listening on, but the two are not crossed over. So you will need to write separate middleware for both Express.js requests chains, and socket.io request chains. There are ways of writing this code once and then writing an adapter to use it across both platforms, but that's not what you've tried to do here.
I would suggest you look at doing just nodejs and express for a time before taking on socket.io as well, otherwise you're trying to learn a whole heap of technologies all at once is quite a lot to try and take on board all at once.
In my node/express app.js main file, I have created a mini-app router:
var router = express.Router();
that I am passing inside to my controller functions and exporting again, at the end, I am registering the router in
app.use('/Link', router);
I now wanted to set up a second Controller folder with extra Controller function and routes only for my Angular NGX-Charts, where I prep up my data from mongoDB in correct format. Therefore, I wanted to create a second router object where I am passing and registering the right routes and middleware for that router object.
My question now is, can I create and register more than one router object for my express instance, like app.use('/Link',router1, router2, router3,...) ?
and does it behave the same like one router object then (I mean, will it find the appropriate router according to which routes I am navigating to in my browser and execute the correct middleware)?
Sure, you can do that. Common use-cases would be password-protection, generating auth tokens, parsing payloads, etc.
app.use accepts any number of "middlewares" after the first argument.
Check the docs for more details: https://expressjs.com/en/4x/api.html#app.use
The arguments are fairly flexible, there are a number of options for what you can pass.
A middleware function.
A series of middleware functions (separated by commas).
An array of middleware functions.
A combination of all of the above.
Each function gets 3 arguments, which are the Request, Response, and next callback function. Here's an example with an inline middleware that logs something and forwards to the next handler.
app.use('/secret-stuff', authorize, (req, res, next) => {
console.log('token from auth middleware', req.authToken)
next()
}, render)
One thing to note is that you can only send one response, so only the final handler would send the response to the user. Everything before that should call next() to activate the next middleware in the chain.
You could pass a number of routers as long as you make sure to forward (call next()) when the paths are unmatched. You would need to use some kind of path pattern that would allow for the middleware routers to handle greater specificity in the path (e.g. '/Link/*'), otherwise you wouldn't be able to define any sub-path handlers in the middleware routers.
In the past, I haven't had the need for sub-routers. Middleware works fine for modularization.
Why does the request object usually denoted as "req", remain the same throughout the application lifecycle? You can see this variable in callbacks with signatures like function(req,res,next)
If I attach a variable to that req object, like req.user, then that user object seems to exist throughout the application, is that true and if so why?
Is this true only of app.use like so?
app.use(function (request, response, next) {
request.user= user;
next();
});
It's not the same "throughout the application lifecycle", it's the same throughout middleware stack. Express is based on Connect, and connect brings the notion of "middleware stack" - you register functions that do something with request/response and able to pass (or not) work up to the next layer in the middleware. All functions in the "stack" operate on the same request/response, but each new request result in new instance of request/response objects.
Basically, the answer is "it's because Connect is designed to work this way"
request object will be same throughtout the request lifecycle and not application lifecycle.
e.g. you have a route like this, with two middlewares (can be of any number)
app.get("/profile", authenticate, UserController.profile);
Now whenever there is a request on path /profile express will call two functions authenticate and UserController.profile in order.
And will pass three arguments to each function req, res, next.
But how will express know if first function has completed its execution? it knows using next callback. So when authenticate will call next() express knows the functions execution is completed and will call UserController.profile with same arguments.
As req is an object and every callback is passed the same req object, so any change you make in the object will be available to every middleware/callback express calls for that particular request.
Is this true only of app.use like so?
No, it is true for routes methods too. Route methods are called only when the route matches, while app.use middlewares are called for each request.
Here are three pieces of terminology used in documentation relating to ConnectJS for NodeJS that keeps getting used, but that I don't completely undertand:
1) views and controllers
2) partials and collections
3) Middleware
Let's start from the bottom up.
Level 0: built-in http module
In the beginning, there is node.js's built-in http.Server written by Ryan Dahl. You write a function(req, res), and Node will call your function each time a new connection is accepted:
// Hello world HTTP server using http module:
var http = require('http');
var app = http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, world.');
});
app.listen(8080, '127.0.0.1');
Level 1: Connect
Connect, written by Tim Caswell, is simply a subclass of http.Server that makes it easier to organize your code. Instead of writing a single callback that handles every request, you chain together some middleware. Each middleware is a function(req, res, next) that handles the request if possible, or calls next(error) if it did not finish handling the user's request. The middleware handlers are called in the order of their use; you should call the catch-all app.use(connect.errorHandler()) at the end.
One important middleware is the router, which allows you to filter some middleware based on a pattern of the URL path. The syntax for the route patterns is based on ruby's Sinatra routes. When I use the filter /hello/:name, req.params.name will be set to the matching part of the URL.
var connect = require('connect');
var app = connect.createServer();
app.use(connect.favicon());
app.use(connect.logger());,
app.use(connect.router(function(app) {
app.get('/hello/:name', function(req, res, next) {
try {
if (Math.random() > 0.5) {
throw new Error('Random error!');
}
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, ' + req.params.name);
} catch (e) {
return next(e);
}
});
}));
app.listen(8080, '127.0.0.1');
In Connect, every handler is middleware! You use whichever functionality you need like bodyParser or cookieParser, and your own business logic is also a middleware function with the same signature function(req, res, next). The connect homepage gives a list of the built-in middleware.
Level 2: Express.js
Express's http server, written by TJ Holowaychuk, is in turn a subclass of Connect that forces the Sinatra style more. In Connect, there was no magic you didn't ask for, but in Express, the router and qs parser (which sets req.query) are automatically used. The router syntax is cleaned up; you call app.get, app.post, etc. directly (and the router is positioned at the first call) rather than putting them inside a function.
Express also contains many other well-documented features and helper functions to extend app, req, and res.
One feature of Express is res.render, which renders the given template file (relative to app.set('views') or $PWD/views) using the template engine implied by the extension, and res.partial, which calls render on each element of a collection (which is just any arraylike object). But I haven't used this optional feature; if you don't care for express's templates you can just res.send data yourself.
Here are some comments. If you have more specific questions, we can try to address them.
1) views and controllers
Views just means a template that can be used to render a response, which is usually HTML but could be plain text or some other format. There are many different templating syntaxes and systems out there. Some work in NodeJS as well as in web browsers. That's all there is to views.
Controllers are the "C" in the MVC design pattern and are responsible as an intermediary between views and models. They are basically the glue that handles some basic things like formatting choices that don't belong in the model code.
2) partials and collections
(Side comment, these are really part of Express.js, not Connect, but they are sibling libraries)
Partials is a document template representing a small portion or snippet of a document, as opposed to a complete HTML document. Partials can be included by other templates and are often re-used by multiple containing templates. Collections go hand in hand with them. For example, you might have a partial to display a "President" object and in that partial you'd have markup for a photo, dates he served as president, political party, etc. You could use that same partial throughout your site whenever you wanted to display a "President" record/object. If you had a collection of several "President" objects, "collections" give you an easy way to say "render a president partial for each president object in this list".
3) middleware
The way connect handles responding to HTTP requests is to route the request through a series of functions called middleware. Each middleware function adheres to a basic API of (req, res, next) and a few behavioral requirements. Each piece of middleware can do one specific bit of processing, then when it's done, call next() to tell connect to move on to the next middleware function in the chain. Connect comes with a bunch of middleware modules which you can see on github. Middleware can do whatever it wants. For example, parse JSON request bodies, search the filesystem for a matching static file to serve, check for session cookies, log to a log file, and so on. This design makes it really easy to re-use code as well as to combine separate middleware functions in novel combinations. Some middleware functions deal with parsing and processing the request, some deal with generating the response. Typically you can find existing middleware functions that do a lot of request processing (parsing, logging, decoding, converting, etc), and you provide your own middleware to actually render the response, which is also usually the last middleware in the chain.