I'm currently using a framework in Node.js ( the botbuilder module from Microsoft Bot Framework) which uses the request[2] module to make HTTP requests.
I'm encountering a problem : this framework seems to send a malformed JSON to Microsoft's servers, but I fail to see why and what is this JSON message made of.
So I'm looking for a way to log those messages, to take a peek at this malformed JSON, as I don't have access to the request object (unless I heavily alter the framework code, which is not something one shall do)
So far, I'm able to log the response body (by adding request to the NODE_DEBUG environment variable), but not the original request body. I did try a tcpdump on our server but since it's all HTTPS there's nothing I can use there.
Any idea / tool that might help ?
Thanks for your time.
Use Node.js middleware to log all your requests. For example, you could use the module request-debug.
Another popular request logging middleware worth knowing about is Morgan, from the Express.js server team.
Related
We are developing a chatbot to handle internal and external processes for a local authority. We are trying to display contact information for a particular service from our api endpoint. The HTTP request is successful and delivers, in part, exactly what we want but there's still some unnecessary noise we can't exclude.
We specifically just want the text out of the response ("Response").
Logically, it was thought all we need to do is drill down into ${dialog.api_response.content.Response} but that fails the HTTP request and ${x.content} returns successful but includes Tags, response and the fields within 1.
Is there something simple we've missed using composer to access what we're after or do we need to change the way our endpoint is responding 2? Unfortunately the MS documentation for FrwrkComp is lacking to say the very least.
n.b. The response is currently set up as a (syntactically) SSML response, this is just a test case using an existing resource.
Response in the Emulator
Snippet from FwrkComp
Turns out it was the first thing I tried just syntactically correct. For the case of the code given it was as simple as:
${dialog.api_response.content[0].Response}
Is it possible to identify the client / library which sent a HTTP request?
I am trying to fetch some data via an API and it is possible to query the API via cURL and python, but when I try to use node (doesn't matter which library, axios requests, unirest, native, ...) or wget I get a proprietary error back from the backend.
Now I am wondering, if the backend is able to identify, which library I am using?
More information:
The requests are exactly the same, so no way to distinguish them
The user-agent header field is set and overwritten for all requests
I already tried to monitor the traffic in wireshark, but couldn't find any differences with the packets on HTTP layer (only the order of the header fields is different, that according to the standard this shouldn't make a difference)
It turns out that the problem was TLS fingerprinting.
See: https://httptoolkit.tech/blog/tls-fingerprinting-node-js/
Nodejs uses google V8 JS engine, V8 based http request clients will not allow you to override headers that would compromise 'web safety', so for example if you are setting "Origin, Host, Referrer" headers, node might refuse to do so. I had the same issue previously.
Un-opinionated http clients, such as the ones written in C++(curl) and python won't 'web safety' check your requests, so that is what is causing the difference in behavior.
In my case I used a C++ library that I called from javascript to make my 'unsafe' requests and the problem was solved.
I’ve to make a post request to a service (not implemented with ServiceStack). From the docs, please correct me if I am wrong, I have to use HTTPUtils nuget package (v. 6.0.2), but if I make a request using its extensions the service returns a 400 bad request.
The same request done using RestSharp (v.105.0) works.
However, I noticed that I had to use an old version compared to the available version of RestSharp.(nothing changes if I downgrade ServiceStack).
Could it be that the service implementation is not compatible with the latest versions of RestSharp and ServiceStack?Is it correct to use HTTPUtils for a service that I don't know if it's implemented with ServiceStack?
Does ServiceStack add some extra wrapper to the .NET framework HTTP client?
Thanks in advance
Here are the docs for ServiceStack's HTTP Utils which can be used for calling generic HTTP APIs, which are extension methods in the ServiceStack.Text NuGet package.
Receiving a 400 Bad Request response suggests that you're sending an invalid request.
Whenever you're investigating issues calling HTTP APIs you should be inspecting the HTTP Traffic with a HTTP tool like WireShark or Fiddler so you can verify that it's sending the HTTP Request you want to send, whilst Postman is a useful tool for quickly working out the HTTP Request you want to send.
If you want help with using a tool you'll need to post the C# source code you're using, the HTTP Request/Response it's sending and the HTTP Request you want to send. Typically the HTTP Response should contain information on why your request is invalid.
A novice javascript developer here!
A have a basic question on whats the best and secured way to make HTTP calls from a front application to a backend service that needs an authentication. My application is a SPA (using Vue.js) & getting data from Java services. Java services need authentication details and return sensitive user data.
I see there are a few options and I wanted to understand a better approach amongst all 3-
Making direct HTTP calls from javascript code- Concern for using this approach is, as Javascript code can also be viewed via dev tools in browser, wont it be easier for anyone to do an inspect and view all critical authentication details hence making overall integration less secure?
Making an HTTP call using Axios via Vue framework- Seems like Axios is Promise based HTTP client for the browser that lets you easily make HTTP calls without much code overhead. but is this secure? is Javascript code loaded in the browser? Or the front end code sends the request and axios makes the request from backend server where the application is hosted?
Using Node- If front end application has unique routes configured for each API call and in my application if I have a route mapping to use request module and node js backend code to make those HTTP calls, is that going to be a robust and secure way of integration?
Please let me know your thoughts and apologies if this is a dumb question!
Not dumb at all. You're just learning.
My first question to your answer 😅 will be: is your application server-side rendered or it's sap + backend?
If it's server-side rendered then I would say it's secured since Node will be sending pages with all required data. On the dev tool, you will only see static files being loaded.
However, if it's SAP, I am not sure whether there is a way to hide whatsoever you send to the server from the dev tool. The only one thing you will need to do is to make sure you encrypt whatever is sensitive to your application.
I have written a web server in nodejs. Most of the time I am receiving a message from one service, doing something, and sending a message to another service. I am in the middle of all the communication.
Sometimes, the communication fails. I am trying to debug what's going on. I would like to examine the request that comes in.
I have a node service, written in express. I have routes, and the routes are passed a req object and a resp object. I should be able to just print out the req object. Problem solved!
But JSON.stringify throws an error. util.inspect doesn't throw an error, but many property values are marked [circular]. The actual property value isn't shown.
When I console.log(req.body) it prints undefined. When I look at req.body using util.inspect, it prints body: {}
I have the feeling the framework is hiding things from me. I don't know how to get the information without it being prettified.
At the tcp/ip level, it's too detailed. At the application level, it's not detailed enough. But at the http level, it should be just right. The request that is received is just text. I should be able to print it out.
I tried using Charles, but I'm having trouble configuring it.
Surely, other people have wanted to see the request as it comes in, before the framework massaged it. How did they do it?
you can use morgan module, it's a HTTP request logger middleware for node.js
I made a more specific question, using a lower level of the node stack of middleware. I got an answer there:
Where did the information I passed in go?
Here is the discussion of how node came to be designed this way:
Node.js - get raw request body using Express
Basically, there used to be a rawBody attribute of the request object in node. People took it out. To accomplish the same thing requires a little bit of code.