Some Connect terminology - node.js

Here are three pieces of terminology used in documentation relating to ConnectJS for NodeJS that keeps getting used, but that I don't completely undertand:
1) views and controllers
2) partials and collections
3) Middleware

Let's start from the bottom up.
Level 0: built-in http module
In the beginning, there is node.js's built-in http.Server written by Ryan Dahl. You write a function(req, res), and Node will call your function each time a new connection is accepted:
// Hello world HTTP server using http module:
var http = require('http');
var app = http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, world.');
});
app.listen(8080, '127.0.0.1');
Level 1: Connect
Connect, written by Tim Caswell, is simply a subclass of http.Server that makes it easier to organize your code. Instead of writing a single callback that handles every request, you chain together some middleware. Each middleware is a function(req, res, next) that handles the request if possible, or calls next(error) if it did not finish handling the user's request. The middleware handlers are called in the order of their use; you should call the catch-all app.use(connect.errorHandler()) at the end.
One important middleware is the router, which allows you to filter some middleware based on a pattern of the URL path. The syntax for the route patterns is based on ruby's Sinatra routes. When I use the filter /hello/:name, req.params.name will be set to the matching part of the URL.
var connect = require('connect');
var app = connect.createServer();
app.use(connect.favicon());
app.use(connect.logger());,
app.use(connect.router(function(app) {
app.get('/hello/:name', function(req, res, next) {
try {
if (Math.random() > 0.5) {
throw new Error('Random error!');
}
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello, ' + req.params.name);
} catch (e) {
return next(e);
}
});
}));
app.listen(8080, '127.0.0.1');
In Connect, every handler is middleware! You use whichever functionality you need like bodyParser or cookieParser, and your own business logic is also a middleware function with the same signature function(req, res, next). The connect homepage gives a list of the built-in middleware.
Level 2: Express.js
Express's http server, written by TJ Holowaychuk, is in turn a subclass of Connect that forces the Sinatra style more. In Connect, there was no magic you didn't ask for, but in Express, the router and qs parser (which sets req.query) are automatically used. The router syntax is cleaned up; you call app.get, app.post, etc. directly (and the router is positioned at the first call) rather than putting them inside a function.
Express also contains many other well-documented features and helper functions to extend app, req, and res.
One feature of Express is res.render, which renders the given template file (relative to app.set('views') or $PWD/views) using the template engine implied by the extension, and res.partial, which calls render on each element of a collection (which is just any arraylike object). But I haven't used this optional feature; if you don't care for express's templates you can just res.send data yourself.

Here are some comments. If you have more specific questions, we can try to address them.
1) views and controllers
Views just means a template that can be used to render a response, which is usually HTML but could be plain text or some other format. There are many different templating syntaxes and systems out there. Some work in NodeJS as well as in web browsers. That's all there is to views.
Controllers are the "C" in the MVC design pattern and are responsible as an intermediary between views and models. They are basically the glue that handles some basic things like formatting choices that don't belong in the model code.
2) partials and collections
(Side comment, these are really part of Express.js, not Connect, but they are sibling libraries)
Partials is a document template representing a small portion or snippet of a document, as opposed to a complete HTML document. Partials can be included by other templates and are often re-used by multiple containing templates. Collections go hand in hand with them. For example, you might have a partial to display a "President" object and in that partial you'd have markup for a photo, dates he served as president, political party, etc. You could use that same partial throughout your site whenever you wanted to display a "President" record/object. If you had a collection of several "President" objects, "collections" give you an easy way to say "render a president partial for each president object in this list".
3) middleware
The way connect handles responding to HTTP requests is to route the request through a series of functions called middleware. Each middleware function adheres to a basic API of (req, res, next) and a few behavioral requirements. Each piece of middleware can do one specific bit of processing, then when it's done, call next() to tell connect to move on to the next middleware function in the chain. Connect comes with a bunch of middleware modules which you can see on github. Middleware can do whatever it wants. For example, parse JSON request bodies, search the filesystem for a matching static file to serve, check for session cookies, log to a log file, and so on. This design makes it really easy to re-use code as well as to combine separate middleware functions in novel combinations. Some middleware functions deal with parsing and processing the request, some deal with generating the response. Typically you can find existing middleware functions that do a lot of request processing (parsing, logging, decoding, converting, etc), and you provide your own middleware to actually render the response, which is also usually the last middleware in the chain.

Related

Whats the difference between a Controller and a Middleware

I am writing and API in express.js. the original API I wrote only utilized routes and raw SQL queries. I have since rewritten the API for the most part NOW using an ORM to react models and migrations.
My question what is the difference and use cases for middleware and controllers. currently only using middleware because most sources online online only explain what a middleware is.
I don't understand the use case of a controller. and I don't want to omit it from my API if its used in proper programming conventions
You should see middleware as a step in your API and controllers as the entity that will actually respond to the requests.
Bellow is an example where authenticationMiddleware is a middleware because it is a step during the processing but should not return the response. It can though, in case of error.
Then getItems actually handle the logic specific to this calls.
As a rule of thumb, middleware are often reused more than once and often they do not response. On contrary, controller respond and are most of the time specific to one endpoint.
const express = require("express");
const app = express();
function authenticationMiddleware(req, res, next) {
// Check that the user is authenticated using req.headers.Authorization
// for example
if (authenticated) {
// The user is authenticated, we can go to the next step
next();
} else {
// The user is not authenticated, we stop here
res.status(401);
res.send("Error during authentication");
}
}
function getItems(req, res, next) {
// Here we focus on the actual response, we assume that the user is authenticated
res.send({ items: [] });
}
app.get("/items", authenticationMiddleware, getItems);
app.post("/items", authenticationMiddleware, createItems); // Re-use the same middleware
app.listen(3000);
If you're referring to the node/express terminology, middleware are simply the callbacks used by the routing functions/methods (get, set, delete, use, etc). Callbacks can either send or not send responses back to the client. The callbacks are really the 'controllers' if you'd like to call them that (eg: ASP.NET Core MVC) but that's up to you. Below is a quote from Express. Note the term 'middleware' and where there's no mention of a 'controller'.
Middleware functions are functions that have access to the request object (req), the response object (res), and the next middleware
function in the application’s request-response cycle. The next
middleware function is commonly denoted by a variable named next.
Middleware functions can perform the following tasks:
Execute any code.
Make changes to the request and the response objects.
End the request-response cycle.
Call the next middleware function in the stack.
'Express' also defines different types of middleware which is useful:
Application-level middleware
Router-level middleware
Error-handling middleware
Built-in middleware Third-party middleware
Here's another nice look at it from Mozilla's pov that does mention a few controller/callback examples.
Beyond that, you can define what a 'controller' is within your team and the naming convention follows from there. Key is your SOLID profile and how you separate your concerns.

Using res.locals in node.js model file

I am overall clueless about how and why you set up a node.js app, and how any of the app.use functions work - the tutorials on it don't explain the why of anything.
Anyway, I have socket.io, res.locals and index.js set up like so in the app.js root file.
const sockets = require('./models/socket')(io)
app.use(function (req, res, next) {
res.locals.user_id = req.session.user_id;
next();
});
const routes = require('./routes/index');
app.use('/', routes);
I'd like to be able to access res.locals in the socket.js model, like I can in index.js found in the routes folder.
I can't guess how to go about doing this. If anybody is able to explain how and why I can or can't that would be a bonus. Thanks!
Welcome to Expressjs, there are a few fundamentals you should probably research before going any further, they'll help solve some of your confusion. I'll give a brief explanation of them but I suggest you do further research. I'll then answer your actual question at the end.
Middleware and app.use
Expressjs is built upon an idea that everything is just "middleware". Middleware is a function which runs as part of a request chain. A request chain is essentially a single client request, which then goes through a chain of a number of middleware functions until it either reaches the end of the chain, exits early by returning a response to the client, or errors.
Express middleware is a function which takes the following three arguments.
req (request) - Representing the request made by a client to your
server.
res (response) - Representing the response you will return to
the client.
next - A way of telling express that your current
middleware function is done, and it should now call the next piece of
middleware. This can either be called "empty" as next(); or with an
error next(new Error());. If it is called empty, it will trigger
the next piece of middleware, if it is called with an error then it
will call the first piece of error middleware. If next is not called at the
end of a piece of middleware, then the request is deemed finished and the
response object is sent to the user.
app.use is a way of setting middleware, this means it will run for every request (unless next() is either not called by the previous piece of middleware for some reason, or it's called with an error). This middleware will run for any HTTP request type (GET, POST, PUT, DELETE, etc).
app.use can take multiple arguments, the important ones for beginners to learn are: app.use(func) and app.use(path, func). The former sets "global" middleware which runs no matter what endpoint (url path) the client requests, the latter (with a specific path) is run only if that specific path is hit. I.e. app.use('/hello', (req, res, next) => { res.send('world'); }); will return "world" when the endpoint "/hello" is hit, but not if the client requests "/hi". Where as app.use((req, res, next) => { res.send('world'); }); would return "world" when you hit any endpoint.
There are more complex things you can do with this, but that's the basics of attaching middleware to your application. The order they are attached to the application, is the order in which they will run.
One more thing, this will blow your mind, an express application made with the standard const app = express() can also be used as middleware. This means you can create several express applications, and then mount them using app.use to a single express application. This is pretty advanced, but does allow you to do some really great things with Express.
Why can you not access res.locals in socket.io? (The real question)
Within your middleware handler, you are setting up a res.locals.use_id property. This only lives with that individual request, you can pass it around as long as the request is alive by passing it into other functions, but outside of that request it doesn't exist. res is literally the response object that tells Express how to respond to the clients request, you can set properties of it during the request but once that HTTP request has ended it's gone.
Socket.io is a way of handling web socket requests, not standard HTTP requests. Thus, in a standard express HTTP request you will not be able to hand off the connection to anything with socket.io, because the connection is a single short lived HTTP request. Likewise, you won't be able to do the same the other way.
If you wish to find the users id in a socket.io request, you'll have to do this within the socket.io request itself.
Right now, you're entering a piece of middleware for an Express.js request, you are then calling next() which runs the next piece of express middleware, at no point does it cross over into Socket.io realms. This is often confused by tutorials because Socket.io can handle requests across the same port as Express is listening on, but the two are not crossed over. So you will need to write separate middleware for both Express.js requests chains, and socket.io request chains. There are ways of writing this code once and then writing an adapter to use it across both platforms, but that's not what you've tried to do here.
I would suggest you look at doing just nodejs and express for a time before taking on socket.io as well, otherwise you're trying to learn a whole heap of technologies all at once is quite a lot to try and take on board all at once.

MEAN: can I register more than a mini router app in app.use()?

In my node/express app.js main file, I have created a mini-app router:
var router = express.Router();
that I am passing inside to my controller functions and exporting again, at the end, I am registering the router in
app.use('/Link', router);
I now wanted to set up a second Controller folder with extra Controller function and routes only for my Angular NGX-Charts, where I prep up my data from mongoDB in correct format. Therefore, I wanted to create a second router object where I am passing and registering the right routes and middleware for that router object.
My question now is, can I create and register more than one router object for my express instance, like app.use('/Link',router1, router2, router3,...) ?
and does it behave the same like one router object then (I mean, will it find the appropriate router according to which routes I am navigating to in my browser and execute the correct middleware)?
Sure, you can do that. Common use-cases would be password-protection, generating auth tokens, parsing payloads, etc.
app.use accepts any number of "middlewares" after the first argument.
Check the docs for more details: https://expressjs.com/en/4x/api.html#app.use
The arguments are fairly flexible, there are a number of options for what you can pass.
A middleware function.
A series of middleware functions (separated by commas).
An array of middleware functions.
A combination of all of the above.
Each function gets 3 arguments, which are the Request, Response, and next callback function. Here's an example with an inline middleware that logs something and forwards to the next handler.
app.use('/secret-stuff', authorize, (req, res, next) => {
console.log('token from auth middleware', req.authToken)
next()
}, render)
One thing to note is that you can only send one response, so only the final handler would send the response to the user. Everything before that should call next() to activate the next middleware in the chain.
You could pass a number of routers as long as you make sure to forward (call next()) when the paths are unmatched. You would need to use some kind of path pattern that would allow for the middleware routers to handle greater specificity in the path (e.g. '/Link/*'), otherwise you wouldn't be able to define any sub-path handlers in the middleware routers.
In the past, I haven't had the need for sub-routers. Middleware works fine for modularization.

Node.js middleware or not

I'm new to Node.js and I'm migrating a simple site of mine to Node.js mostly as a learning experience.
In all my sites, I like to keep the most relevant information on the site in a "sitemeta" object. This is queried from Redis on each request and if that fails (which it only does if sitemeta gets updated and then it reads it from MySQL instead and saves it in redis so that next request again gets it from redis as redis is much speedier than MySQL).
So, in PHP I would simply add the call for sitemeta in a settings.php file that I always include on top of each file so that the information like home_url or site_mode (that are part of sitemeta object) is always there.
However, now in Node.js I was wondering if this is really the way to go or if there's a better way to actually have this happen as a middleware rather than making like so, at the top of every controller file (router files, really).
//in index.js
var express = require('express');
var router = express.Router();
var site = require('./../lib/sitemeta');
...
//using it for something.
var siteMeta = site.meta();
res.render('index', { title: siteMeta.title });
Or, could I even have the sitemeta instantiated in app.js so that it's queried only once (when the node app.js starts) unless it needs to be updated and if so a refresh can be provoked somehow?
Thanks.
I think the best approach is through a middleware. Wherever you store you metadata (memory, redis, etc), having a middleware will let you inject you siteMeta in all requests, with the capacity to adapt your siteMeta based on the received request (locale, etc). We usually use a lot of small, easily testable middleware functions to inject different data elements for another middleware down the pipeline to process and produce the response.
// with promises
app.use(function(req, res, next){
req.siteMeta = loadSiteMeta();
next();
});
app.get('/endpoint', function(req, res){
// wait for the promise to be fulfilled
req.siteMeta.then(function(siteMeta){
res.render('my-view', { title: siteMeta.title });
});
});
// without promises
app.use(function(req, res, next){
loadSiteMeta(function(err, siteMeta){
req.siteMeta = siteMeta;
next();
});
});
app.get('/endpoint', function(req, res) {
// siteMeta will be populated
res.render('my-view', { title: req.siteMeta.title });
});
The actual implementation of loadSiteMeta will depend on your selected storage. The benefits of using promises instead of classic callbacks here is that, if you have multiple middleware loading different data elements before reaching your final processing function, they will be performed in parallel instead of sequentially. You might want to use Promise.all() to wait for all promises you need in your final function.

How does mounting apps in Express work

So I've seen TJ's guide to creating modular Express-apps, followed it to good effects, but want to know more about the details of how it works, however a search gives me no answers.
In short I am wondering: When mounting apps in Express, what parts of the apps are shared and what parts are not?
Some examples to clarify my question:
app.js:
app.use(express.bodyParser());
app.use(loginApi); //loginApi is an express app
app.listen(3000);
This example works. But if I place the app.use(loginApi) before app.use(express.bodyParser()); , the body parser will not be available in the loginApi subapp. Why is that?
Another example:
submodule.js
var app = module.exports = require('express')();
app.all('*', function(req, res, next){
console.log('nifty middleware');
next();
});
app.js
app.get('/funtimes', fn);
app.use(submodule);
app.listen(3000);
Now in this example, If I understand it correctly, the /funtimes route will not be affected by the submodule middleware for all routes. But what about the rest of the routes of app.js ? Will they be affected? And what if I add another module, will it be affected?
if I place the app.use(loginApi) before app.use(express.bodyParser()); , the body parser will not be available in the loginApi subapp. Why is that?
That's because of the way Express handles requests. Any incoming request starts at the top of the middleware stack, starting with app.use() stack.
Middleware are simply functions that have the function signature function(req, res, next) which either call next() if they want to hand off the request to subsequent functions, or send a response themselves. You define a 'middleware chain' of a bunch of these functions (many are provided by express, like express.logger() and express.compress().)
So in the following scenario:
app.use(express.bodyParser());
var loginApi = require('./api/index.js')
app.use(loginApi);
app.use(app.router);
then an incoming request will first hit app.use(express.bodyParser()), parsing req.body. Then that function calls its internal next(), passing it to the next function in the middleware chain (app.use(loginApi)). The loginApi app has its own middleware chain, but the requests already have req.body set from the outer app. If the loginApi doesn't send a response, the request will continue to app.use(app.router) and at that point the requests will go to the routing functions you set up in your outer app.
So the answer is: A mounted app will have the middleware functions shared, which are placed before the line app.use(loginApi)
Middleware runs in order (until one of the middlewares doesn't call next()).
If you use() your mounted app before use()ing bodyParser, the entire sub-app will run before bodyParser adds its properties.
What you're asking about is middleware. This confused me for a while. Middleware are the functions that run in order to take a request in and serve back a response. app.use() takes a function as its only argument. That function manipulates the request in a consistent way.
app.use is a lot like app.all("*").
The order matters. For example, you might want to run a validator before serving the response.
One thing I learned recently is that you can pass an array of middleware functions to a route. For example
app.get("/whatever",[
function(req,res,next}{
...validation...
next();
},
function(req,res) {
...actions....
res.send(200);
}
]);
The next callback tells express to run the next function in the middleware. Middleware can also modify the request object. This is used a lot in authentication. For example, you'll see req.user getting defined from the database so in later middleware you'll be able to refer to properties of the user. But, it can also be used for a ton of other stuff.

Resources