I have an app where the Nginx reverse proxy is making a lot of requests to my health endpoint. I'd like to not log these at all so my output logs are smaller. I'm also logging everything as JSON using Network.Wai.Middleware.RequestLogger.JSON, which has a function to format log messages as JSON.
One thing I could do is log an empty bytestring, but I was thinking there may be some way to no-op the log call. I can't figure out how to do that from looking at the various RequestLogger functions in wai-extra.
Does anyone have a recommendation for how to build a custom Middleware in order to not-log certain requests?
I created a custom formatter in the following way:
-- | Wai Application Middleware logger
jsonRequestLogger :: IO Middleware
jsonRequestLogger = mkRequestLogger
$ def { outputFormat = CustomOutputFormatWithDetails dontLogHealthEndpoint }
dontLogHealthEndpoint :: OutputFormatterWithDetails
dontLogHealthEndpoint date req status responseSize duration reqBody response =
if B.isInfixOf "health" $ rawPathInfo req
then toLogStr B.empty
else formatAsJSON date req status responseSize duration reqBody response
This seems to work fine. However, I'd still like to know if there's a better way.
Related
Context
I am using EasyCron (https://www.easycron.com) to ping an endpoint in my IHP app.
If the code on my app's side fails for some reason, I would like to set the HTTP status code to 503 or similar and display the errors in the response so that,
the cron job is marked as failed on EasyCron and
I can see the relevant IHP errors in EasyCron's cron job records.
Since my response is not going to be parsed by EasyCron, I don't need it to be in JSON format. It's just going to be saved in the logs.
Question
I see that there is renderJsonWithStatusCode but since I have no need to parse the JSON and it is extra work to create JSON instances of my data, I would prefer to render a plain text response but with a custom status code. Is this possible?
You can set a custom status code by calling IHP's respondAndExit with a custom response:
import Network.Wai (responseLBS)
import Network.HTTP.Types (status503)
import Network.HTTP.Types.Header (hContentType)
action MyAction = do
let text = "An error happend"
respondAndExit $ responseLBS status503 [(hContentType, "text/plain")] text
I am new to the whole backend stuff I understood that both bodyparser and express.json() will parse the incoming request(body from the client) into the request object.
But what happens if I do not parse the incoming request from the client ?
without middleware parsing your requests, your req.body will not be populated. You will then need to manually go research on the req variable and find out how to get the values you want.
Your bodyParser acts as an interpreter, transforming http request, in to an easily accessible format base on your needs.
You may read more on HTTP request here ( You can even write your own http server )
https://nodejs.org/api/http.html#http_class_http_incomingmessage
You will just lose the data, and request.body field will be empty.
Though the data is still sent to you, so it is transferred to the server, but you have not processed it so you won't have access to the data.
You can parse it yourself, by the way. The request is a Readable stream, so you can listen data and end events to collect and then parse the data.
You shall receive what you asked for in scenarios where you do not convert the data you get the raw data that looks somewhat like this username=scott&password=secret&website=stackabuse.com, Now this ain't that bad but you will manually have to filter out which is params, what is a query and inside of those 2 where is the data..
unless it is a project requirement all that heavy lifting is taken care of by express and you get a nicely formatted object looking like this
{
username: 'scott',
password: 'secret',
website: 'stackabuse.com'
}
For Situation where you DO need to use the raw data express gives you a convenient way of accessing that as well all you need to do is use this line of code
express.raw( [options] ) along with express.json( [options] )
every time i do an http request to any route in my app i got in my console the route method , execution time and and anther number that i have no idea what it means(will love to know) .
does anyone know how can i use this data? i want to record routes that takes more then x time so i can improve those routes.
i try do find from console. in the express framework as i assumed its coming from there but no luck
As I can see from the screenshot, you're probably using morgan middleware.
By default it logs all the requests in the dev format which is -
:method :url :status :response-time ms - :res[content-length]
So according to your screenshot -
Patch is the method.
/trades/ is the url path.
200 is the status code.
5.893 is the response time.
2 is content length of the response.
I think you were asking about content length. It's basically the size of your response body in bytes.
Looking at the source, Morgan keeps track of the request start / end times by adding instance properties to the request.
The request time looks like it's calculated when the response starts to write headers, as such you would need to follow the same pattern....but only after Morgan has been configured e.g.
import onHeaders from 'on-headers';
import logger from 'morgan';
app.use(logger()); // configure morgan
app.use((req, res, next) =>
onHeaders(res, () => {
console.log(logger['response-time'](req, res)); // this may work
// alternatively, use req._startAt / res._startAt to work it out (see source for implementation)
});
return next();
)
The other alternative of course is just do it yourself with custom middleware, or use something off the shelf like response-time
I realise outq is used to see the last 100 or so responses for processed messages. However, the objects stored in outq only seem to have the response body, not the originating request, so it can be quite difficult to debug issues.
Is there an easy way to automatically include a copy of the originating inq message as well?
I've found a solution that works. Not sure if its optimal, but it seems to do the job. When defining the handler I just create a new response object and insert original request into it.
mqService.RegisterHandler<MyRequest>(
m => {
var response = ObjectFactory.GetInstance<MyService>().Post((MyRequest) m.Body);
return new {result = response, request = m.Body};
}
I am developing an application using Scotty and of course WAI. I would like to be able to limit the size of requests, both for body length and for headers. How can I do that? Is it possible to do it using a plain WAI middleware ?
I don't know details of Scotty, but it's certainly possible to set up a WAI middleware that will look at the requestBodyLength and, if it's too large, return an appropriate 413 status code page. One thing you'd need to deal with is if the upload body is sent with chunked encoding, in which case no content-length is present. but that's uncommon. You have the option of either rejecting those requests, or adding code to wrap the request body and return an error if it turns out to be too large (that's what Yesod does).
The marked solution points in the correct direction, but if you're like me you might still struggle to explicitely derive the full code needed. Here is an implementation (thanks to the help of an experienced Haskell friend):
import qualified Network.HTTP.Types as Http
import qualified Network.Wai as Wai
limitRequestSize :: Wai.Middleware
limitRequestSize app req respond = do
case Wai.requestBodyLength req of
Wai.KnownLength len -> do
if len > maxLen
then respond $ Wai.responseBuilder Http.status413 [] mempty
else app req respond
Wai.ChunkedBody ->
respond $ Wai.responseBuilder Http.status411 [] mempty
where
maxLen = 50*1000 -- 50kB
The middleware then just runs in scotty's do block like this
import Network.Wai.Middleware.RequestLogger (logStdout)
main :: IO ()
main = do
scotty 3000 $ do
middleware logStdout
middleware limitRequestSize
get "/alive" $ do
status Http.status200
-- ...
If you're curious as to how to derive it (or why I found this not overly trivial), consider that Middleware is an alias for
Application -> Application
where Application itself is an alias for
Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
Hence there are quite a bunch of arguments to (mentally) unpack, even if the solution is pretty terse.
As of wai-extra-3.1.1 the code described above has been added to the Network.Wai.Middleware.RequestSizeLimit module, so it can just be pulled in as a dependency.