I was going through the NestJs Docs. And there's this image.
https://docs.nestjs.com/pipes
Filters are more oriented towards Client-Side and pipes are towards Controllers. To me both seems similar.
What are the differences between Pipe and filter with their respective common use-cases ?
Pipes are meant to consuming incoming data from the request, be it url or query parameters, or a request body, and do some validations and/or transformations on them to ensure they are the shape your server expects them to be. Nest has some built in utilities like the ValidationPipe to help with this.
Filters (AKA Exception Filters) on the other hand are meant for catching errors that happened during the execution of a request and handling it, sending the error back to the client in a nice format, taking care of sending back the proper error codes, and any other error handling logic you have (like possibly sending to a monitoring service). Nest has a built in ExceptionFilter that manages this nicely, but you can always create your own to handle the logic differently.
Related
relatively new to Node.js and I've found that there's a lot of support for multiple callback functions. I'd love some help in understanding the applications of having such a structure (why isn't one enough?) Thanks!
Might it have something to do with making async requests in our callback functions?
i.e. I'm taking handling a get request, and must first make a get request of my own before ultimately returning the request to the client.
What's the difference between an Interceptor, Filter and Middleware in Nest.js framework? When should one of them be used and favored over the other?
Thanks
As you already implied with your question, all three are very similar concepts and in a lot of cases it is hard to decide and comes down to your preferences. But I can give an overview of the differences:
Interceptors
Interceptors have access to response/request before and after the route handler is called.
Registration
Directly in the controller class with #UseInterceptors() controller- or method-scoped
Globally with app.useGlobalInterceptors() in main.ts
Examples
LoggingInterceptor: Request before route handler and afterwards its result. Meassure time it takes.
ResultMapping: Transform null to [] or wrap result in a response object: users -> {users: users}
Conclusion
I like that the registration is closer to the route handlers compared to middleware. But there are some limitations, for example, you cannot set the response code or alter the response with Interceptors when you send the response with the library-specific #Res() object in your route handler, see docs.
Middleware
Middleware is called only before the route handler is called. You have access to the response object, but you don't have the result of the route handler. They are basically express middleware functions.
Registration
In the module, very flexible way of choosing relevant routes (with wildcards, by method,...)
Globally with app.use() in main.ts
Examples
FrontendMiddleware: redirect all routes except API to index.html, see this thread
You can use any express middleware that is out there. There are lots of libraries, e.g. body-parser or morgan
Conclusion
The registration of middleware is very flexible, for example: apply to all routes but one etc. But since they are registered in the module, you might not realize it applies to your controller when you're looking at its methods. It's also great that you can make use of all the express middleware libraries that are out there.
Exception Filters
Exception Filters are called after the route handler and after the interceptors. They are the last place to make changes before a response goes out.
Registration
Directly in the controller class with #UseFilters() controller- or method-scoped
Globally app.useGlobalFilters() in your main.ts
Examples
UnauthorizedFilter: Map to an easy to understand message for the user
NotFoundFilter: Map all routes that are not found (not part of your api) to your index.html.
Conclusion
The basic use case for exception filters are giving understandable error messages (hiding technical details). But there are also other creative ways of usage: When you serve a single page application, then typically all routes should redirect to index.html except the routes of your API. Here, you can redirect on a NotFoundException. Some might find this clever others hacky. Your choice. ;-)
So the execution order is:
Middleware -> Interceptors -> Route Handler -> Interceptors -> Exception Filter (if exception is thrown)
With all three of them, you can inject other dependencies (like services,...) in their constructor.
For those of us who "get it" better visually, I've created this NestJs pipeline digram based on the latest v6.10 version. Please feel free to point out any inaccuracies. I'll review and update it promptly, if needed.
I'm assuming that you mean Pipes instead of Filters as Filters are primarily tied to Exception Handling.
There is definitely some overlap as Middleware are a flexible way of composing any web application but are more of a generic concept (creating a stack of functions to build a pipeline). The others are Nest specific concepts and as such tie in a bit more naturally with things like Dependency Injection.
Pipes are used to transform input data (and optionally to do validation).
Interceptors are really neat because they can transform both data coming in and leaving your API. They give you the ability to mutate what the original handler would have returned through the use of observable streams. This is something that you would probably need to implement using two middlewares (on either side of the handler).
Use Pipes when you want to transform data coming in to a handler.
Use Interceptors when bi-directional transformation is required.
Use middlewares when you want to stick closer to the traditional (eg Express) way of building your web app or when you want to more broadly apply functionality to many handlers at once (there's less decorators floating around in your code).
Middleware
The usage and function of the middleware of Nest.js is basically equivalent to Express.js, which generally functions for:
execute any code.
make changes to the request and the response objects.
end the request-response cycle.
call the next middleware function in the stack.
if the current middleware function does not end the request-response
cycle, it must call next() to pass control to the next middleware
function. Otherwise, the request will be left hanging.
Interceptor
The interceptor can transform data between the controller and the client-side, which can perform functions for:
bind extra logic before / after method execution
transform the result returned from a function
transform the exception thrown from a function
extend the basic function behavior
completely override a function depending on specific conditions
(e.g., for caching purposes)
Filter
In Nest.js, the Filter generally refers to the Exception filter, which is responsible for processing all unhandled exceptions across an application:
a built-in global exception filter handles exceptions of type HttpException (and subclasses of it)
custom exception filters are designed for controlling the exact flow of control and the content of the response sent back to the client. For example, add logging or use a different JSON schema based on some dynamic factors.
I'm fairly new to programming and this question is about making sure I get the HTTP protocol correctly. My issue is that when I read about HTTP request/response, it looks like it needs to be in a very specific format with a status code, HTTP version number, headers, a blank line followed by the body.
However, after creating a web app with nodejs/express, I never once had to actually write code that made an HTTP response in this format (I'm assuming, although I don't know for sure that other frameworks like ruby on rails or python/Django are the same). In the express app, I just set up the route handlers to render the appropriate pages, when a request was made to that route.
Is this because express is actually putting the response in the correct HTTP format behind the scenes? In other words, if I looked at the expressJS code, would there be something in that code that actually makes an HTTP response in the HTTP format?
My confusion is that, it seems like the HTTP request/response format is so important but somehow I never had to write any code dealing with it for a node/express application. Maybe this is the entire point of a framework like express... to take out the details so that developers can deal with business logic. And if that is correct, does anyone ever write web apps without a framework to do this. Would you then be responsible for writing code that puts the server's response into the exact HTTP format?
I'm fairly new to programming and this question is about making sure I get the HTTP protocol correctly. My issue is that when I read about HTTP request/response, it looks like it needs to be in a very specific format with a status code, HTTP version number, headers, a blank line followed by the body.
Just to give you an idea, there are probably hundreds of specifications that have something to do with the HTTP protocol. They deal with not only the protocol itself, but also with the data format/encoding for everything you send including headers and all the various content types you can send, authentication schemes, caching, status codes, URL decoding, etc.... You can see some of the specifications involved just by looking here: https://www.w3.org/Protocols/.
Now a simple request and a simple text response could get away with only knowing a few of these specifications, but life is not always that simple.
Is this because express is actually putting the response in the correct HTTP format behind the scenes? In other words, if I looked at the expressJS code, would there be something in that code that actually makes an HTTP response in the HTTP format?
Yes, there would. A combination of Express and the HTTP library that is built into node.js handle all the details of the specification for you. That's the advantage of using a library/framework. They even handle different versions of the protocol and feedback from thousands of other developers have helped them to clean up edge case bugs. A good library/framework allows you to still control any detail about the response (headers, content types, status codes, etc..) without making you have to go through the detail work of actually creating the exact response. This is a good thing. It lets you write code faster and lets you ride on the shoulders of others who have already figured out minutiae details that have nothing to do with the logic of your app.
In fact, one could say the same about the TCP protocol below the HTTP protocol. No regular app developer wants to write their own TCP stack. Instead, you just want a working TCP stack that you can use that's already been tuned and debugged for you.
However, after creating a web app with nodejs/express, I never once had to actually write code that made an HTTP response in this format (I'm assuming, although I don't know for sure that other frameworks like ruby on rails or python/Django are the same). In the express app, I just set up the route handlers to render the appropriate pages, when a request was made to that route.
Yes, this is a good thing. The framework did the detail work for you. You just call res.setHeader(), res.status(), res.cookie(), res.send(), res.json(), etc... and Express makes the entire response for you.
And if that is correct, does anyone ever write web apps without a framework to do this. Would you then be responsible for writing code that puts the server's response into the exact HTTP format?
If you didn't use a framework or library of any kind and were programming at the raw TCP level, then yes you would be responsible for all the details of the HTTP protocol. But, hardly anybody other than library developers ever does this because frankly it's just a waste of time. Every single platform has at least one open source library that does this already and even if you were working on a brand new platform, you could go get an open source body of code and port it to your platform much quicker than you could write all this yourself.
Keep in mind that one of the HUGE advantages of node.js is that there's an enormous body of open source code (mostly in NPM and Github) already prepackaged to work with node.js. And, because node.js is server-side where code memory isn't usually tight and where code just comes from the local hard disk at server init time, there's little downside to grabbing a working and tested package that does what you already need, even if you're only going to use 5% of the functionality in the package. Or, worst case, clone an existing repository and modify it to perfectly suit your needs.
Is this because express is actually putting the response in the
correct HTTP format behind the scenes?
Yes, exactly, HTTP is so ubiquitous that almost all programming languages / frameworks handle the actual writing and parsing of HTTP behind the scenes.
Does anyone ever write web apps without a framework to do this. Would
you then be responsible for writing code that puts the server's
response into the exact HTTP format?
Never (unless you're writing code that needs very low level tweaking of HTTP code or something)
Does anyone have any information on where the HTTP Status Codes (200, 404, 500, etc) are first available in the IIS Pipeline? I'm trying to write a series of http modules and handlers for error handling purposes and I don't particularly want to duplicate requests/responses to get the values.
Take a look at any of the events here, and take your pick :)
http://msdn.microsoft.com/en-us/library/ms693685(v=vs.90).aspx
Theoretically, the status code can be changed by any of the http modules in the pipeline; it just depends on which events they are subscribed to.
For example, an authorization module may subscribe to the OnAuthorizeRequest method, and perform its logic at that time, and change the status code if needed. In another case, a classic ASP app may run as a handler, and you won't be able to determine if the status code is 500 until OnPostExecuteRequestHandler. Finally, an error in a logging module may generate a 500, which doesn't occur until the request processing is nearly finished (OnLogRequest)
Further complicating matters, some handlers may spit out unbuffered data during execution, so it could be in any of the OnSendResponse events, which don't come in any particular order, and the status code could have changed between responses.
So, it really depends on what you're trying to achieve in order to approach this effectively. If you could provide more detail, perhaps we could formulate a solution.
(I'm not familiar to RESTFul, please correct me if my concept is wrong)
In RESTFul architecture, we map every action to an URL. If I click "post a article", may it's actually URL http://example.com/ and some data action=post&content=blahblah.
If I want to post, but not refresh the whole web page, I can use javascript's XMLHTTPRequest. I post it and then get it's content and insert it to a div in my page. These action is all asynchronous.
Then I know there is something named WebSocket and it's wrapper socket.io. It use "message" to communicate between client and server. When I click "post" the client just call socket.send(data) and wait for server's client.send(data). It's magical. But how about URL?
It's possible to use the two model both without repeating myself? In other word, every action has it's URL, and some of them can interact with user real-timely(by socket.io?)
Moreover, should I do this? In a very interactive web program(ex. games), the RESTFul is still meaningful?
You're defining a handler for actions that map to REST over http. POST and GET generally refer to update and query over an entity. There's absolutely no reason you can't just define a handler for generic versions of these CRUD operations that can be used in both contexts. The way I generally do this is by introducing the concept of a 'route' to the real-time transport, and mapping those back to the same CRUD handlers.
You have a session, you can impose the same ACL, etc.
+---------------------------------+
| |
| BROWSER |
| |
+--+--^-------------------+---^---+
| | | |
| | | |
+--v--+---+ +--v---+---+
| | | |
| HTTP | | SOCKET.IO|
+--+---^--+ +--+---^---+
| | | |
+--v---+------------------v---+---+
| |
| ROUTING/PUBSUB |
+-+--^-------+--^-------+--^------+
| | | | | |
+-v--+--+ +-v--+--+ +-v--+-+
| | | | | |
| USERS | | ITEMS | |ETC |
+-------+ +-------+ +------+
ENTITY CRUD HANDLERS
I posted this on my blog recently:
Designing a CRUD API for WebSockets
When building Weld, we are using both REST and WebSockets (Socket.io). Three observations on WebSockets:
Since WebSockets are so free-form, you can name events how you want but it will eventually be impossible to debug.
WebSockets don’t have the request/response form of HTTP so sometimes it can be difficult to tell where an event is coming from, or going to.
It would be nice if the WebSockets could fit into the existing MVC structure in the app, preferably using the same controllers as the REST API.
My solution:
I have two routing files on my server: routes-rest.js and routes-sockets.js
My events look like this example: "AppServer/user/create".
I use forward slashes (“/”) to make the events look like routing paths.
The first string is the target (~”host name” if this actually was a path).
The second string is the model.
The third string is the CRUD verb: i.e. create, read, update, delete.