Ive built an app that pipes requests through express validator's validation chains, but I ran into a design issue related to logging and error handling.
My app is divided into distinct controllers/services/repositories. All errors thrown in repos and layers bubble up to controllers which handle errors by calling next() on them and passing them to an error handling middleware that logs and so on.
The problem is that one of my validation chains calls a repo. If my DB connection is dead, all I get on my log is what I happen to put in the validation chain's withMessage().
Id like to get a better log of the event in such case, but now a dead db connection ends in a 400 Bad request on the logs since there isnt a controller to catch the error.
Any ideas how I should structure my app to combat this?
I would not like to add specific logging on the service/data layers only because I use express validator as a middleware.
I looked more into the responsibilities of each layer and came to the conclusion that the call to the data layer should not be done with express-validator. Logic related to whether an email is already registered, for example, should happen on the service layer, as per:
In which layer should validation be performed?
I will use express validator only for easy checks and put the rest into my service layer.
Related
I have a microservices-based architecture in place.
Service A has details about Cart and the Cart model looks like this:
Cart {
Id,
Items,
Price,
UserId
}
Service B has User details. User model:
User {
Id,
FirstName,
LastName,
Email
}
I want to fetch these User details from Service A by communicating with Service B. I am trying to implement this communication between microservices Asynchronously using a messaging queue. The problem I am facing is how to associate a particular message with the corresponding HTTP request from the client?
Is it a good idea to handle client HTTP requests with asynchronous communication among the services?
To be more specific, how do I associate the message received in step 5 in the above image with the HTTP request handler in step 1.
Service Bus works great when your two processes are disconnected completely, things like downstream processing of an order after it is placed for example. You want the extra layer of certainty that the downstream process will run even if it takes a retry or two without having to wait for it. Trying to use it for a case where you need an immediate response is going to cause the issues you are seeing- that's not what it is for.
The best solution is going to be something along the lines of having your cart service call the user service over http so that the data stays properly correlated. If you want to keep a layer in between to remove the direct dependency between the two services, something like API Management is going to be a much better fit.
Hi,
We have a NestJS application that is reachable both by HTTP and gRPC (using a custom RPC strategy).
Whilst they don't share the same routes, they do share some code. For example, a service to lookup resources from the database. We initially followed the documentation and used Exceptions filters to manage HTTP response status code. This means that service throws an instance of NotFoundException when a resource cannot be found.
However now that we're trying to integrate with the RPC exception handler, we found that any type of exception that isn't a instance of RpcException will be considered a Internal Server Error by the RPC microservice, including HttpExceptions.
After looking at the documentation and the source code I cannot find a way to share exceptions correctly between those two microservices, but I can be totally mistaken.
Is there a way to share code between HTTP and RPC services while still reporting exceptions accurately through both protocol?
You can bind a global exception filter / or interceptor to the gRPC microservice, that would transform every thrown HttpException. At the moment there's no generic exception class for each transport type.
Replying to my own question: based on reported issue, this is not supported when in the context of an Hybrid Application.
My project requirements have changed however so I am unlikely to investigate further.
I am working on logs of the application just to give you information there are two kind of strategies used inside the application.
POST Process
PRE Process
In the first strategy if application has to make some call to third-party applications, it will first send the response to client and after execute that call, so
as my application is using express as framework, I can catch those responses in middle-wares but not in this specific case application is not sending back
any response as application has already responded to the client.
The second strategy is simple process and at the end send back response this call will be caught by express middle-ware without any issue, so that is the
model that i am using until now now what I want to do is to catch requests received, and sent by application by standing outside the application, as i
got the idea and understand the structure this is kind of a proxy server which will catch requests not just received by the application but sent by as well, and i know we can catch requests coming in but I am working on the data at runtime
so i don't want those logs i want logs that are coming into the logger application at runtime.
Now coming to the question according to my requirements is there a way to catch requests received, and sent by NodeJS server?
can you try morgan node module for that
Create a new morgan logger middleware function using the given format and options. The format argument may be a string of a predefined name (see below for the names), a string of a format string, or a function that will produce a log entry.
https://www.npmjs.com/package/morgan
I just started trying out SailsJS a few days ago.
I've realized that the Node is terminated whenever I have an uncaught exception.
I have a list of controllers and each of them calls a specific service JS file (Containing logics and DB calls) in services/.
Can I write a global error handler for all services so that any type of error that occurs from these services should be handled by it and appropriate error response has to be communicated to front-end.
I tried using process.on('uncaughtexception') or some of basic exceptions but it needs to be added to each service method.
Also can I have one common point for all service calls made from client to server through which all io.socket.post() and io..socket.get() goes through
I would appreciate any pointer/article that would show me the common best practices for handling uncaught exceptions in SailsJS and using shorter code rather than writing redundant code in all services.
Best practice is using Domains in your controller. This will handle exceptions in async code, and its relatively straight forward.
You can use something like trycatch to simplify things a little, but domain based exceptions will be most effective. It'll insure that exceptions do not crash your application. Just create a new domain in your controller and run your controller methods inside of that domain.
Sailsjs being based on express you can use connect middleware, and you can seamlessly create a new domain from middleware. There such thing as express-domain-middleware. This might be the most aesthetic option, and most convenient.
Update:
As mention by Benjamin Gruenbaum, Domains are planned to become deprecated in v1 of node. Perhaps you should read through Joyents Error Handling Best Practices. Its agnostic to the framework you are using.
Additonally you can still use Domains, while there isn't a way to globally handle errors in node.js otherwise. Once deprecated you could always remove your dependence on Domains, relatively easily. That said, it may be best not to rely solely on domains.
Strongloop also provides a library inspired by domains called Zone. This is also an option.
Its OK to let node instance error out due to a programming error, else it may continue in an inconsistent state and mess-up business logic. In production environment the server can be restarted on crash, this will reset its state and keep it available, if the error is not frequent. And in all of it its very important to Log everything. This applies to most of Node setups, including SailsJS.
The following approach can be taken:
Use a Logger : A dedicated logger should be accessible to server components. Should be connected to a service that notifies the developer (email ?) of very serious errors.
Propagate per request errors to the end: Carefully forward errors from any step in request processing. In ExperssJs/ConnectJs/middle-ware based setup's, the next(err) can be used to pass an error to the middle-ware chain. An error catching middle-ware at the end of the chain will get this error, log it verbose, and send a 500 status back. You can use Domains or Zones or Promises or async or whatever you like to process request and catch errors.
Shutdown on process.on('uncaughtexception'): Log the erorr, do necessary clean-up, and throw the same error again to shutdown process.
User PM2/Forever or Upstart/init.d on linux : Now when the process shuts down due to the bad exception, these tools will restart it and track how many time server has been crashing. If the server is crashing way too many time, its good to stop it and take immediate action.
I have not tried this, but I believe you should be able to set a catch-all exception handler in bootstrap.js using process.on('uncaughtexception').
Personally, I use promises via the bluebird library, and put a catch statement that passes all errors to a global error handling function.
Inside a nodejs connect middleware, the default way of error reporting is to call next(err), usually followed by return, if no further message should be shown to the user. The error handler may show an vanilla http 500 page then for example.
However, some errors may result in exceptions, including those throwed by used third party libs. The connect (or express?) middleware stack however catches those, and redirect them to the error handler as well.
I followed some discussions saying nodejs should be restarted on exceptions, as some state may be corrupted. However, the connect (or express) makers doesn't share this view it seems?
Having it this way, is it feasable to just throw exceptions inside middleware? Or may this baypass some connect-internal operation?